[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:18 dist-test-slave-0t40 krb5kdc[25857](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:18 dist-test-slave-0t40 krb5kdc[25857](info): set up 2 sockets
Jun 19 20:36:18 dist-test-slave-0t40 krb5kdc[25857](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 20:36:19.277401 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:36:19 dist-test-slave-0t40 krb5kdc[25857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:36:19.396901 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:37973
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:37973
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:19.476810 25870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:19.477006 25870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:19.477028 25870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:19.478188 25870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:19.478227 25870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:19.478242 25870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:19.478255 25870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:19.478266 25870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:19.479655 25870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:37973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:37973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:19.479846 25870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:19.480008 25870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:19.485162 25876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:19.485291 25870 server_base.cc:1034] running on GCE node
W20260619 20:36:19.485142 25875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:19.485127 25878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:19.485707 25870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:19.485977 25870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:19.487146 25870 hybrid_clock.cc:648] HybridClock initialized: now 1781901379487129 us; error 32 us; skew 500 ppm
Jun 19 20:36:19 dist-test-slave-0t40 krb5kdc[25857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901379, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:19.489769 25870 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:36:19.490139 25870 webserver.cc:460] Webserver started at http://127.25.62.190:41791/ using document root <none> and password file <none>
I20260619 20:36:19.490334 25870 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:19.490372 25870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:19.490458 25870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:19.491531 25870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "b8e8a7f28c3b448bbb379910cf7c4df3"
format_stamp: "Formatted at 2026-06-19 20:36:19 on dist-test-slave-0t40"
server_key: "e6e5c4b48d53c6a3620830c5b78d3e77"
server_key_iv: "52d11a323932e22dffde5c25b8cac122"
server_key_version: "encryptionkey@0"
I20260619 20:36:19.491824 25870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "b8e8a7f28c3b448bbb379910cf7c4df3"
format_stamp: "Formatted at 2026-06-19 20:36:19 on dist-test-slave-0t40"
server_key: "e6e5c4b48d53c6a3620830c5b78d3e77"
server_key_iv: "52d11a323932e22dffde5c25b8cac122"
server_key_version: "encryptionkey@0"
I20260619 20:36:19.493089 25870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:36:19.493829 25885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:19.494045 25870 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:19.494139 25870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "b8e8a7f28c3b448bbb379910cf7c4df3"
format_stamp: "Formatted at 2026-06-19 20:36:19 on dist-test-slave-0t40"
server_key: "e6e5c4b48d53c6a3620830c5b78d3e77"
server_key_iv: "52d11a323932e22dffde5c25b8cac122"
server_key_version: "encryptionkey@0"
I20260619 20:36:19.494199 25870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:19.519207 25870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:19.521442 25870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:19.521557 25870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:19.525447 25870 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:37973
I20260619 20:36:19.525503 25937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:37973 every 8 connection(s)
I20260619 20:36:19.525858 25870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:19.526751 25938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:19.529000 25938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3: Bootstrap starting.
I20260619 20:36:19.529621 25938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:19.529917 25938 log.cc:824] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:19.530761 25938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3: No bootstrap required, opened a new log
I20260619 20:36:19.531781 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 25870
I20260619 20:36:19.531862 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:36:19.532495 25850 external_mini_cluster.cc:1428] Setting key cccfee9ea779ec8948221aef9da7145d
I20260619 20:36:19.532872 25938 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e8a7f28c3b448bbb379910cf7c4df3" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37973 } }
I20260619 20:36:19.533072 25938 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:19.533123 25938 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8e8a7f28c3b448bbb379910cf7c4df3, State: Initialized, Role: FOLLOWER
I20260619 20:36:19.533259 25938 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e8a7f28c3b448bbb379910cf7c4df3" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37973 } }
I20260619 20:36:19.533347 25938 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:19.533378 25938 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:19.533416 25938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:19.534423 25938 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e8a7f28c3b448bbb379910cf7c4df3" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37973 } }
I20260619 20:36:19.534518 25938 leader_election.cc:302] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8e8a7f28c3b448bbb379910cf7c4df3; no voters:
I20260619 20:36:19.534770 25938 leader_election.cc:288] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:19.534791 25943 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:19.534898 25943 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3 [term 1 LEADER]: Becoming Leader. State: Replica: b8e8a7f28c3b448bbb379910cf7c4df3, State: Running, Role: LEADER
I20260619 20:36:19.534976 25943 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e8a7f28c3b448bbb379910cf7c4df3" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37973 } }
I20260619 20:36:19.535022 25938 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:19.535283 25943 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8e8a7f28c3b448bbb379910cf7c4df3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e8a7f28c3b448bbb379910cf7c4df3" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37973 } } }
I20260619 20:36:19.535351 25943 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:19.535465 25943 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8e8a7f28c3b448bbb379910cf7c4df3. Latest consensus state: current_term: 1 leader_uuid: "b8e8a7f28c3b448bbb379910cf7c4df3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8e8a7f28c3b448bbb379910cf7c4df3" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37973 } } }
I20260619 20:36:19.535529 25943 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:19.535663 25948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:19.536273 25948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:19.537967 25948 catalog_manager.cc:1347] Generated new cluster ID: f0cfd75462c046648a7548a0b9f61ba0
I20260619 20:36:19.538030 25948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:36:19 dist-test-slave-0t40 krb5kdc[25857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:19.548139 25942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:19.533838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47438 (local address 127.25.62.190:37973)
0619 20:36:19.534092 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:19.534103 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:19.534143 (+ 40us) server_negotiation.cc:407] Connection header received
0619 20:36:19.536115 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:19.536121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:19.536182 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:19.536326 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:19.538467 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:19.539416 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:19.540482 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:19.540908 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:19.541080 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:19.544398 (+ 3318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:19.544402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:19.544402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:19.544415 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:19.546210 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:19.546674 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:19.546677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:19.546678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:19.546778 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:19.547067 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:19.547068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:19.547069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:19.547315 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:19.547510 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:19.547769 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 20:36:19.547899 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":136,"threads_started":1}
I20260619 20:36:19.549865 25948 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:19.550509 25948 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:19.555202 25948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8e8a7f28c3b448bbb379910cf7c4df3: Generated new TSK 0
I20260619 20:36:19.555351 25948 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
I20260619 20:36:19.583294 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:37973
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:19.664031 25964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:19.664237 25964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:19.664264 25964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:19.665488 25964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:19.665529 25964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:19.665555 25964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:36:19.666990 25964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:37973
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:19.667243 25964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:19.667469 25964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:19.672359 25972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:19.672689 25969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:19.672698 25970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:19.673086 25964 server_base.cc:1034] running on GCE node
I20260619 20:36:19.673308 25964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:19.673584 25964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:19.674753 25964 hybrid_clock.cc:648] HybridClock initialized: now 1781901379674741 us; error 24 us; skew 500 ppm
Jun 19 20:36:19 dist-test-slave-0t40 krb5kdc[25857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901379, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:19.677486 25964 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:36:19.677942 25964 webserver.cc:460] Webserver started at http://127.25.62.129:34947/ using document root <none> and password file <none>
I20260619 20:36:19.678125 25964 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:19.678165 25964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:19.678251 25964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:19.679217 25964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "a9b40fdcf9024054beb3fa9ddd735109"
format_stamp: "Formatted at 2026-06-19 20:36:19 on dist-test-slave-0t40"
server_key: "d1fd3356d9e8314096fea314ec85f550"
server_key_iv: "1d982c6b37ee7c1cf006fd8986128816"
server_key_version: "encryptionkey@0"
I20260619 20:36:19.679505 25964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "a9b40fdcf9024054beb3fa9ddd735109"
format_stamp: "Formatted at 2026-06-19 20:36:19 on dist-test-slave-0t40"
server_key: "d1fd3356d9e8314096fea314ec85f550"
server_key_iv: "1d982c6b37ee7c1cf006fd8986128816"
server_key_version: "encryptionkey@0"
I20260619 20:36:19.680719 25964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:36:19.681494 25979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:19.681674 25964 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:36:19.681762 25964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "a9b40fdcf9024054beb3fa9ddd735109"
format_stamp: "Formatted at 2026-06-19 20:36:19 on dist-test-slave-0t40"
server_key: "d1fd3356d9e8314096fea314ec85f550"
server_key_iv: "1d982c6b37ee7c1cf006fd8986128816"
server_key_version: "encryptionkey@0"
I20260619 20:36:19.681830 25964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:19.693415 25964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:19.695423 25964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:19.695571 25964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:19.695820 25964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:19.696138 25964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:19.696173 25964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:19.696194 25964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:19.696244 25964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:19.702310 25964 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:46263
I20260619 20:36:19.702798 25964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:36:19.703385 26092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:46263 every 8 connection(s)
I20260619 20:36:19.707926 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 25964
I20260619 20:36:19.708026 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:36:19.708215 25850 external_mini_cluster.cc:1428] Setting key fbd7197cf3c21b6abcd4893ec6afdf7a
Jun 19 20:36:19 dist-test-slave-0t40 krb5kdc[25857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901379, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:19.712890 25942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:19.703681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:57093 (local address 127.25.62.190:37973)
0619 20:36:19.703778 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:19.703781 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:19.703965 (+ 184us) server_negotiation.cc:407] Connection header received
0619 20:36:19.704518 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:19.704519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:19.704534 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:19.704586 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:19.705617 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:19.706127 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:19.707150 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:19.707358 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:19.707454 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:19.709950 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:19.709951 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:19.709952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:19.709960 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:19.711496 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:19.711908 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:19.711909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:19.711910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:19.711972 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:19.712231 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:19.712232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:19.712232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:19.712349 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:19.712430 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:19.712713 (+ 283us) server_negotiation.cc:299] Negotiation successful
0619 20:36:19.712800 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 20:36:19.713137 26095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:19.703631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37973 (local address 127.25.62.129:57093)
0619 20:36:19.703820 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:19.703870 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:19.704383 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:19.704652 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:19.704656 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:19.705311 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:19.705511 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:19.705516 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:19.706244 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:19.706244 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:19.707036 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:19.707038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:19.707494 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:19.707495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:19.707608 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:19.708206 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:19.708216 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:19.709846 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:19.711580 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:19.711587 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:19.711590 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:19.711810 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:19.712043 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:19.712044 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:19.712044 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:19.712154 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:19.712421 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:19.712428 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:19.712573 (+ 145us) client_negotiation.cc:769] Sending connection context
0619 20:36:19.712690 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 20:36:19.712845 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":89,"threads_started":1}
I20260619 20:36:19.713721 26093 heartbeater.cc:347] Connected to a master server at 127.25.62.190:37973
I20260619 20:36:19.713817 26093 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:19.714012 26093 heartbeater.cc:511] Master 127.25.62.190:37973 requested a full tablet report, sending...
I20260619 20:36:19.714578 25902 ts_manager.cc:194] Registered new tserver with Master: a9b40fdcf9024054beb3fa9ddd735109 (127.25.62.129:46263)
I20260619 20:36:19.715221 25902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:57093
WARNING: no policy specified for kudu/127.25.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
I20260619 20:36:19.739262 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:37973
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:19.820731 26098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:19.820935 26098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:19.820958 26098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:19.822106 26098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:19.822152 26098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:19.822180 26098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:36:19.823747 26098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:37973
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:19.823925 26098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:19.824096 26098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:19.829128 26103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:19.829111 26106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:19.829169 26104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:19.829432 26098 server_base.cc:1034] running on GCE node
I20260619 20:36:19.829674 26098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:19.829962 26098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:19.831175 26098 hybrid_clock.cc:648] HybridClock initialized: now 1781901379831160 us; error 25 us; skew 500 ppm
Jun 19 20:36:19 dist-test-slave-0t40 krb5kdc[25857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901379, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:19.833782 26098 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:36:19.834257 26098 webserver.cc:460] Webserver started at http://127.25.62.130:39111/ using document root <none> and password file <none>
I20260619 20:36:19.834501 26098 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:19.834580 26098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:19.834751 26098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:19.835932 26098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "d349b8f2fdc844b9975d4dd67e9940db"
format_stamp: "Formatted at 2026-06-19 20:36:19 on dist-test-slave-0t40"
server_key: "09ec69c0ca19e7925e3dfdea13ed7e70"
server_key_iv: "1cc691c469751a5459f4b4f7f204a4fb"
server_key_version: "encryptionkey@0"
I20260619 20:36:19.836319 26098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "d349b8f2fdc844b9975d4dd67e9940db"
format_stamp: "Formatted at 2026-06-19 20:36:19 on dist-test-slave-0t40"
server_key: "09ec69c0ca19e7925e3dfdea13ed7e70"
server_key_iv: "1cc691c469751a5459f4b4f7f204a4fb"
server_key_version: "encryptionkey@0"
I20260619 20:36:19.837698 26098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260619 20:36:19.838431 26113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:19.838635 26098 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:19.838729 26098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "d349b8f2fdc844b9975d4dd67e9940db"
format_stamp: "Formatted at 2026-06-19 20:36:19 on dist-test-slave-0t40"
server_key: "09ec69c0ca19e7925e3dfdea13ed7e70"
server_key_iv: "1cc691c469751a5459f4b4f7f204a4fb"
server_key_version: "encryptionkey@0"
I20260619 20:36:19.838819 26098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:19.850759 26098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:19.852633 26098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:19.852797 26098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:19.853013 26098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:19.853369 26098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:19.853404 26098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:19.853425 26098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:19.853438 26098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:19.859444 26098 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:40397
I20260619 20:36:19.859488 26226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:40397 every 8 connection(s)
I20260619 20:36:19.859782 26098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:36:19.864300 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 26098
I20260619 20:36:19.864374 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:36:19.864547 25850 external_mini_cluster.cc:1428] Setting key 23c643eae033cdb87417d7c039c7545a
Jun 19 20:36:19 dist-test-slave-0t40 krb5kdc[25857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901379, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:19.869568 25942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:19.860419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:38993 (local address 127.25.62.190:37973)
0619 20:36:19.860463 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:19.860465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:19.860707 (+ 242us) server_negotiation.cc:407] Connection header received
0619 20:36:19.861305 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:19.861306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:19.861318 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:19.861371 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:19.862303 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:19.862806 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:19.863806 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:19.864002 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:19.864093 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:19.866482 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:19.866484 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:19.866485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:19.866493 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:19.867874 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:19.868335 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:19.868336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:19.868336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:19.868401 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:19.868693 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:19.868694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:19.868694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:19.868841 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:19.868903 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:19.869305 (+ 402us) server_negotiation.cc:299] Negotiation successful
0619 20:36:19.869425 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:36:19.869652 26229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:19.860392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37973 (local address 127.25.62.130:38993)
0619 20:36:19.860574 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:19.860632 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:19.861198 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:19.861439 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:19.861442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:19.862016 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:19.862193 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:19.862197 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:19.862907 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:19.862909 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:19.863691 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:19.863693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:19.864123 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:19.864125 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:19.864246 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:19.864779 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:19.864787 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:19.866358 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:19.867990 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:19.867997 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:19.868000 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:19.868239 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:19.868473 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:19.868474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:19.868474 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:19.868593 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:19.868905 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:19.868912 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:19.869114 (+ 202us) client_negotiation.cc:769] Sending connection context
0619 20:36:19.869253 (+ 139us) client_negotiation.cc:240] Negotiation successful
0619 20:36:19.869426 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":87,"threads_started":1}
I20260619 20:36:19.870575 26227 heartbeater.cc:347] Connected to a master server at 127.25.62.190:37973
I20260619 20:36:19.870695 26227 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:19.870870 26227 heartbeater.cc:511] Master 127.25.62.190:37973 requested a full tablet report, sending...
I20260619 20:36:19.871306 25902 ts_manager.cc:194] Registered new tserver with Master: d349b8f2fdc844b9975d4dd67e9940db (127.25.62.130:40397)
I20260619 20:36:19.871696 25902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:38993
WARNING: no policy specified for kudu/127.25.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
I20260619 20:36:19.892148 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:37973
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:19.972656 26232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:19.972862 26232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:19.972886 26232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:19.973995 26232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:19.974031 26232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:19.974057 26232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:36:19.975472 26232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:37973
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:19.975660 26232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:19.975831 26232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:19.981137 26240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:19.981216 26238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:19.981137 26237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:19.981614 26232 server_base.cc:1034] running on GCE node
I20260619 20:36:19.981818 26232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:19.982054 26232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:19.983227 26232 hybrid_clock.cc:648] HybridClock initialized: now 1781901379983212 us; error 30 us; skew 500 ppm
Jun 19 20:36:19 dist-test-slave-0t40 krb5kdc[25857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901379, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:19.985962 26232 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:36:19.986441 26232 webserver.cc:460] Webserver started at http://127.25.62.131:41089/ using document root <none> and password file <none>
I20260619 20:36:19.986685 26232 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:19.986758 26232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:19.986914 26232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:19.987955 26232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "65fc384a7a6d40a4ba2d2aa8a66f3580"
format_stamp: "Formatted at 2026-06-19 20:36:19 on dist-test-slave-0t40"
server_key: "f25e3cb5f04dbc1968c13180ce597a47"
server_key_iv: "83de8e95f1328782f653e2e89ddb43b9"
server_key_version: "encryptionkey@0"
I20260619 20:36:19.988333 26232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "65fc384a7a6d40a4ba2d2aa8a66f3580"
format_stamp: "Formatted at 2026-06-19 20:36:19 on dist-test-slave-0t40"
server_key: "f25e3cb5f04dbc1968c13180ce597a47"
server_key_iv: "83de8e95f1328782f653e2e89ddb43b9"
server_key_version: "encryptionkey@0"
I20260619 20:36:19.989745 26232 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:36:19.990506 26247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:19.990687 26232 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:36:19.990790 26232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "65fc384a7a6d40a4ba2d2aa8a66f3580"
format_stamp: "Formatted at 2026-06-19 20:36:19 on dist-test-slave-0t40"
server_key: "f25e3cb5f04dbc1968c13180ce597a47"
server_key_iv: "83de8e95f1328782f653e2e89ddb43b9"
server_key_version: "encryptionkey@0"
I20260619 20:36:19.990876 26232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:20.013213 26232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:20.015549 26232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:20.015750 26232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:20.016008 26232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:20.016566 26232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:20.016651 26232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:20.016693 26232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:20.016729 26232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:20.022743 26232 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:43793
I20260619 20:36:20.022794 26360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:43793 every 8 connection(s)
I20260619 20:36:20.023104 26232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:36:20.027011 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 26232
I20260619 20:36:20.027092 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:36:20.027312 25850 external_mini_cluster.cc:1428] Setting key d874169fda67963342eb1baae473506d
Jun 19 20:36:20 dist-test-slave-0t40 krb5kdc[25857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901379, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:20.032455 25942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:20.023799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:55417 (local address 127.25.62.190:37973)
0619 20:36:20.023864 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:20.023867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:20.024062 (+ 195us) server_negotiation.cc:407] Connection header received
0619 20:36:20.024602 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:20.024603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:20.024617 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:20.024663 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:20.025672 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:20.026164 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:20.027239 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:20.027486 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:20.027574 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:20.029901 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:20.029902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:20.029903 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:20.029909 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:20.031257 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:20.031623 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:20.031624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:20.031624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:20.031677 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:20.031908 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:20.031908 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:20.031908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:20.032013 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:20.032069 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:20.032310 (+ 241us) server_negotiation.cc:299] Negotiation successful
0619 20:36:20.032390 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:36:20.032579 26363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:20.023784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37973 (local address 127.25.62.131:55417)
0619 20:36:20.023927 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:20.023972 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:20.024488 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:20.024730 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:20.024734 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:20.025367 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:20.025563 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:20.025567 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:20.026265 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:20.026266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:20.027093 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:20.027096 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:20.027583 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:20.027584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:20.027740 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:20.028325 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:20.028335 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:20.029806 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:20.031331 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:20.031336 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:20.031337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:20.031552 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:20.031739 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:20.031739 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:20.031740 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:20.031835 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:20.032073 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:20.032077 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:20.032183 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 20:36:20.032262 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 20:36:20.032371 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":66,"threads_started":1}
I20260619 20:36:20.033107 26361 heartbeater.cc:347] Connected to a master server at 127.25.62.190:37973
I20260619 20:36:20.033195 26361 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:20.033363 26361 heartbeater.cc:511] Master 127.25.62.190:37973 requested a full tablet report, sending...
I20260619 20:36:20.033735 25902 ts_manager.cc:194] Registered new tserver with Master: 65fc384a7a6d40a4ba2d2aa8a66f3580 (127.25.62.131:43793)
I20260619 20:36:20.034143 25902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:55417
I20260619 20:36:20.039407 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 20:36:20 dist-test-slave-0t40 krb5kdc[25857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901380, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 20:36:20 dist-test-slave-0t40 krb5kdc[25857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901380, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.62.129@KRBTEST.COM
I20260619 20:36:20.062036 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:20.054201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34766 (local address 127.25.62.129:46263)
0619 20:36:20.054367 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:20.054377 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:20.054386 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:36:20.054409 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:20.054411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:20.054424 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:20.054520 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:20.055096 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:20.055912 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:20.056676 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:20.057011 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:20.057137 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:20.059294 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:20.059302 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:20.059303 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:20.059310 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:20.060754 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:20.061165 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:20.061168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:20.061168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:20.061231 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:20.061445 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:20.061446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:20.061446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:20.061588 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:20.061774 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:20.061818 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 20:36:20.061896 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":60,"threads_started":1}
W20260619 20:36:20.062399 26007 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34766
Jun 19 20:36:20 dist-test-slave-0t40 krb5kdc[25857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 20:36:20 dist-test-slave-0t40 krb5kdc[25857](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.129@KRBTEST.COM
I20260619 20:36:20.084796 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:20.077390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34780 (local address 127.25.62.129:46263)
0619 20:36:20.077454 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:20.077456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:20.077540 (+ 84us) server_negotiation.cc:407] Connection header received
0619 20:36:20.077627 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:20.077628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:20.077645 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:20.077714 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:20.078301 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:20.078789 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:20.079557 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:20.079762 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:20.079880 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:20.082280 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:20.082282 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:20.082282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:20.082289 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:20.083718 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:20.084063 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:20.084064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:20.084065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:20.084115 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:20.084369 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:20.084370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:20.084370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:20.084491 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:20.084603 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:20.084670 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 20:36:20.084733 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 20:36:20.085440 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 25964
I20260619 20:36:20.089975 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 26098
I20260619 20:36:20.094204 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 26232
I20260619 20:36:20.098872 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 25870
2026-06-19T20:36:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1879 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:20 dist-test-slave-0t40 krb5kdc[26375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:20 dist-test-slave-0t40 krb5kdc[26375](info): set up 2 sockets
Jun 19 20:36:20 dist-test-slave-0t40 krb5kdc[26375](info): commencing operation
krb5kdc: starting...
W20260619 20:36:21.140789 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:36:21 dist-test-slave-0t40 krb5kdc[26375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:36:21.263561 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:41785
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:45425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:41785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:21.344693 26388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:21.344880 26388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:21.344909 26388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:21.346061 26388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:21.346103 26388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:21.346118 26388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:21.346131 26388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:21.346143 26388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:21.347568 26388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:45425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:41785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:41785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:21.347813 26388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:21.348064 26388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:21.353648 26396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:21.353644 26394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:21.353706 26393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:21.353942 26388 server_base.cc:1034] running on GCE node
I20260619 20:36:21.354176 26388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:21.354473 26388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:21.355666 26388 hybrid_clock.cc:648] HybridClock initialized: now 1781901381355671 us; error 52 us; skew 500 ppm
Jun 19 20:36:21 dist-test-slave-0t40 krb5kdc[26375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901381, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:21.358346 26388 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:36:21.358857 26388 webserver.cc:460] Webserver started at http://127.25.62.190:41393/ using document root <none> and password file <none>
I20260619 20:36:21.359115 26388 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:21.359195 26388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:21.359354 26388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:21.360452 26388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "84e963cc18fb4b0caba42221231c2a89"
format_stamp: "Formatted at 2026-06-19 20:36:21 on dist-test-slave-0t40"
server_key: "c286576e39164d5012f748bcecadb73d"
server_key_iv: "ee0d6e3241d1b655a4697d6fd4bd9d39"
server_key_version: "encryptionkey@0"
I20260619 20:36:21.360845 26388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "84e963cc18fb4b0caba42221231c2a89"
format_stamp: "Formatted at 2026-06-19 20:36:21 on dist-test-slave-0t40"
server_key: "c286576e39164d5012f748bcecadb73d"
server_key_iv: "ee0d6e3241d1b655a4697d6fd4bd9d39"
server_key_version: "encryptionkey@0"
I20260619 20:36:21.362430 26388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:21.363564 26403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:21.363742 26388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:21.363848 26388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "84e963cc18fb4b0caba42221231c2a89"
format_stamp: "Formatted at 2026-06-19 20:36:21 on dist-test-slave-0t40"
server_key: "c286576e39164d5012f748bcecadb73d"
server_key_iv: "ee0d6e3241d1b655a4697d6fd4bd9d39"
server_key_version: "encryptionkey@0"
I20260619 20:36:21.363963 26388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:21.388152 26388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:21.390223 26388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:21.390440 26388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:21.394970 26388 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:41785
I20260619 20:36:21.395015 26455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:41785 every 8 connection(s)
I20260619 20:36:21.395385 26388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:21.396288 26456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:21.398537 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 26388
I20260619 20:36:21.398588 26456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89: Bootstrap starting.
I20260619 20:36:21.398625 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:36:21.398768 25850 external_mini_cluster.cc:1428] Setting key e8ac7d44133c677a38dd6296c6879d17
I20260619 20:36:21.399297 26456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:21.399636 26456 log.cc:824] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:21.400444 26456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89: No bootstrap required, opened a new log
I20260619 20:36:21.402066 26456 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e963cc18fb4b0caba42221231c2a89" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41785 } }
I20260619 20:36:21.402199 26456 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:21.402220 26456 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84e963cc18fb4b0caba42221231c2a89, State: Initialized, Role: FOLLOWER
I20260619 20:36:21.402351 26456 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e963cc18fb4b0caba42221231c2a89" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41785 } }
I20260619 20:36:21.402422 26456 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:21.402451 26456 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:21.402498 26456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:21.403295 26456 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e963cc18fb4b0caba42221231c2a89" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41785 } }
I20260619 20:36:21.403416 26456 leader_election.cc:302] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84e963cc18fb4b0caba42221231c2a89; no voters:
I20260619 20:36:21.403568 26456 leader_election.cc:288] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:21.403657 26461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:21.403805 26456 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:21.403860 26461 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89 [term 1 LEADER]: Becoming Leader. State: Replica: 84e963cc18fb4b0caba42221231c2a89, State: Running, Role: LEADER
I20260619 20:36:21.403960 26461 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e963cc18fb4b0caba42221231c2a89" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41785 } }
I20260619 20:36:21.404414 26461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84e963cc18fb4b0caba42221231c2a89. Latest consensus state: current_term: 1 leader_uuid: "84e963cc18fb4b0caba42221231c2a89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e963cc18fb4b0caba42221231c2a89" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41785 } } }
I20260619 20:36:21.404412 26462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84e963cc18fb4b0caba42221231c2a89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84e963cc18fb4b0caba42221231c2a89" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41785 } } }
I20260619 20:36:21.404526 26461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:21.404543 26462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:21.404793 26468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:21.405325 26468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:21.407018 26468 catalog_manager.cc:1347] Generated new cluster ID: 80ee9203d5674269814de9d1f498655f
I20260619 20:36:21.407068 26468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:36:21 dist-test-slave-0t40 krb5kdc[26375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:21.412295 26459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:21.399332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36496 (local address 127.25.62.190:41785)
0619 20:36:21.399514 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:21.399522 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:21.399549 (+ 27us) server_negotiation.cc:407] Connection header received
0619 20:36:21.400178 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:21.400192 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:21.400245 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:21.400368 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:21.401157 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.402721 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.404068 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.404870 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.405020 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:21.408086 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:21.408090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:21.408091 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:21.408103 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:21.410194 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:21.410633 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:21.410636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:21.410636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:21.410742 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:21.410982 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:21.410983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:21.410983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:21.411284 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:21.411443 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:21.411934 (+ 491us) server_negotiation.cc:299] Negotiation successful
0619 20:36:21.412082 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260619 20:36:21.414378 26468 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:21.415100 26468 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:21.421602 26468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84e963cc18fb4b0caba42221231c2a89: Generated new TSK 0
I20260619 20:36:21.421816 26468 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
I20260619 20:36:21.453631 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41785
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:45425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:21.534188 26482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:21.534422 26482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:21.534461 26482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:21.535645 26482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:21.535722 26482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:21.535768 26482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:36:21.537331 26482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:45425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41785
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:21.537600 26482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:21.537827 26482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:21.542860 26487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:21.542908 26490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:21.543052 26488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:21.543197 26482 server_base.cc:1034] running on GCE node
I20260619 20:36:21.543346 26482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:21.543605 26482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:21.544749 26482 hybrid_clock.cc:648] HybridClock initialized: now 1781901381544734 us; error 28 us; skew 500 ppm
Jun 19 20:36:21 dist-test-slave-0t40 krb5kdc[26375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901381, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:21.547302 26482 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:36:21.547762 26482 webserver.cc:460] Webserver started at http://127.25.62.129:38291/ using document root <none> and password file <none>
I20260619 20:36:21.548002 26482 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:21.548072 26482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:21.548218 26482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:21.549400 26482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "64ce00011a7144b480b51b4d1edef726"
format_stamp: "Formatted at 2026-06-19 20:36:21 on dist-test-slave-0t40"
server_key: "5ac5821683e4e7a5e3e3872290805e0a"
server_key_iv: "17d8ecb1a4636e7f511dc5c6ffb1830f"
server_key_version: "encryptionkey@0"
I20260619 20:36:21.549773 26482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "64ce00011a7144b480b51b4d1edef726"
format_stamp: "Formatted at 2026-06-19 20:36:21 on dist-test-slave-0t40"
server_key: "5ac5821683e4e7a5e3e3872290805e0a"
server_key_iv: "17d8ecb1a4636e7f511dc5c6ffb1830f"
server_key_version: "encryptionkey@0"
I20260619 20:36:21.551100 26482 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 20:36:21.551885 26497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:21.552115 26482 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:21.552215 26482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "64ce00011a7144b480b51b4d1edef726"
format_stamp: "Formatted at 2026-06-19 20:36:21 on dist-test-slave-0t40"
server_key: "5ac5821683e4e7a5e3e3872290805e0a"
server_key_iv: "17d8ecb1a4636e7f511dc5c6ffb1830f"
server_key_version: "encryptionkey@0"
I20260619 20:36:21.552295 26482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:21.574666 26482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:21.577008 26482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:21.577190 26482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:21.577418 26482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:21.577730 26482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:21.577785 26482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:21.577824 26482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:21.577852 26482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:21.583467 26482 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:34221
I20260619 20:36:21.583555 26610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:34221 every 8 connection(s)
I20260619 20:36:21.583849 26482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:36:21.588932 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 26482
I20260619 20:36:21.589051 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:36:21.589236 25850 external_mini_cluster.cc:1428] Setting key 70efa83ca9cecd8fc9c9ad08baaa7420
Jun 19 20:36:21 dist-test-slave-0t40 krb5kdc[26375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901381, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:21.593210 26459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:21.584636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:35565 (local address 127.25.62.190:41785)
0619 20:36:21.584704 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:21.584706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:21.584908 (+ 202us) server_negotiation.cc:407] Connection header received
0619 20:36:21.585812 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:21.585813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:21.585823 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:21.585882 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:21.586436 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.586929 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.587877 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.588077 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.588127 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:21.590279 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:21.590280 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:21.590281 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:21.590288 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:21.591724 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:21.592210 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:21.592211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:21.592211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:21.592260 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:21.592546 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:21.592547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:21.592547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:21.592651 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:21.592705 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:21.593024 (+ 319us) server_negotiation.cc:299] Negotiation successful
0619 20:36:21.593147 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:36:21.593464 26613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:21.584616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:41785 (local address 127.25.62.129:35565)
0619 20:36:21.584776 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:21.584817 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:21.585712 (+ 895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:21.585938 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:21.585942 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:21.586178 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:21.586359 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:21.586363 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.587014 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.587015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:21.587799 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:21.587801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.588151 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.588152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:21.588246 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:21.588771 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:21.588779 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:21.590180 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:21.591817 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:21.591823 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:21.591825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:21.592134 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:21.592329 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:21.592330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:21.592330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:21.592466 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:21.592722 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:21.592727 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:21.592889 (+ 162us) client_negotiation.cc:769] Sending connection context
0619 20:36:21.593001 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 20:36:21.593193 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260619 20:36:21.594048 26611 heartbeater.cc:347] Connected to a master server at 127.25.62.190:41785
I20260619 20:36:21.594151 26611 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:21.594323 26611 heartbeater.cc:511] Master 127.25.62.190:41785 requested a full tablet report, sending...
I20260619 20:36:21.594858 26420 ts_manager.cc:194] Registered new tserver with Master: 64ce00011a7144b480b51b4d1edef726 (127.25.62.129:34221)
I20260619 20:36:21.595461 26420 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:35565
WARNING: no policy specified for kudu/127.25.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
I20260619 20:36:21.617501 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41785
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:45425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:21.697736 26616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:21.697918 26616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:21.697942 26616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:21.699018 26616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:21.699052 26616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:21.699079 26616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:36:21.700421 26616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:45425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41785
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:21.700590 26616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:21.700752 26616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:21.705735 26621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:21.705742 26624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:21.705963 26616 server_base.cc:1034] running on GCE node
W20260619 20:36:21.705765 26622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:21.706357 26616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:21.706599 26616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:21.707764 26616 hybrid_clock.cc:648] HybridClock initialized: now 1781901381707753 us; error 28 us; skew 500 ppm
Jun 19 20:36:21 dist-test-slave-0t40 krb5kdc[26375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901381, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:21.710145 26616 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:36:21.710597 26616 webserver.cc:460] Webserver started at http://127.25.62.130:35317/ using document root <none> and password file <none>
I20260619 20:36:21.710824 26616 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:21.710892 26616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:21.711017 26616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:21.712080 26616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "f5e5c4b5c2814d15857ffecdda66f275"
format_stamp: "Formatted at 2026-06-19 20:36:21 on dist-test-slave-0t40"
server_key: "427bc99ecf67447ec99dec045e40f04e"
server_key_iv: "159bc4232101aa519fd8f6c0b446ea48"
server_key_version: "encryptionkey@0"
I20260619 20:36:21.712423 26616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "f5e5c4b5c2814d15857ffecdda66f275"
format_stamp: "Formatted at 2026-06-19 20:36:21 on dist-test-slave-0t40"
server_key: "427bc99ecf67447ec99dec045e40f04e"
server_key_iv: "159bc4232101aa519fd8f6c0b446ea48"
server_key_version: "encryptionkey@0"
I20260619 20:36:21.713784 26616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:36:21.714548 26631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:21.714758 26616 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:21.714859 26616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "f5e5c4b5c2814d15857ffecdda66f275"
format_stamp: "Formatted at 2026-06-19 20:36:21 on dist-test-slave-0t40"
server_key: "427bc99ecf67447ec99dec045e40f04e"
server_key_iv: "159bc4232101aa519fd8f6c0b446ea48"
server_key_version: "encryptionkey@0"
I20260619 20:36:21.714942 26616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:21.728442 26616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:21.730387 26616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:21.730558 26616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:21.730786 26616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:21.731076 26616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:21.731135 26616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:21.731175 26616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:21.731200 26616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:21.737002 26616 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:44359
I20260619 20:36:21.737071 26744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:44359 every 8 connection(s)
I20260619 20:36:21.737357 26616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:36:21.742096 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 26616
I20260619 20:36:21.742198 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:36:21.742367 25850 external_mini_cluster.cc:1428] Setting key 6851e3b4e54d6e54e3b7c62e746ada64
Jun 19 20:36:21 dist-test-slave-0t40 krb5kdc[26375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901381, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:21.746884 26459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:21.738118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:56001 (local address 127.25.62.190:41785)
0619 20:36:21.738195 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:21.738197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:21.738392 (+ 195us) server_negotiation.cc:407] Connection header received
0619 20:36:21.739369 (+ 977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:21.739370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:21.739379 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:21.739435 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:21.740013 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.740497 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.741572 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.741761 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.741815 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:21.744000 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:21.744002 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:21.744003 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:21.744011 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:21.745502 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:21.745932 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:21.745934 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:21.745934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:21.745994 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:21.746260 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:21.746261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:21.746262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:21.746370 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:21.746452 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:21.746719 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 20:36:21.746797 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 20:36:21.747112 26747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:21.738090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:41785 (local address 127.25.62.130:56001)
0619 20:36:21.738266 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:21.738307 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:21.739266 (+ 959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:21.739502 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:21.739505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:21.739746 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:21.739932 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:21.739936 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.740582 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.740583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:21.741493 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:21.741495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.741838 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.741839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:21.741932 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:21.742493 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:21.742501 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:21.743911 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:21.745606 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:21.745613 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:21.745615 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:21.745834 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:21.746068 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:21.746069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:21.746069 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:21.746181 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:21.746442 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:21.746448 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:21.746584 (+ 136us) client_negotiation.cc:769] Sending connection context
0619 20:36:21.746697 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 20:36:21.746847 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
I20260619 20:36:21.747753 26745 heartbeater.cc:347] Connected to a master server at 127.25.62.190:41785
I20260619 20:36:21.747828 26745 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:21.748047 26745 heartbeater.cc:511] Master 127.25.62.190:41785 requested a full tablet report, sending...
I20260619 20:36:21.748472 26420 ts_manager.cc:194] Registered new tserver with Master: f5e5c4b5c2814d15857ffecdda66f275 (127.25.62.130:44359)
I20260619 20:36:21.748852 26420 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:56001
WARNING: no policy specified for kudu/127.25.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
I20260619 20:36:21.768807 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41785
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:45425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:21.850339 26750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:21.850545 26750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:21.850571 26750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:21.851652 26750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:21.851691 26750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:21.851720 26750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:36:21.853266 26750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:45425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41785
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:21.853451 26750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:21.853622 26750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:36:21.858922 26750 server_base.cc:1034] running on GCE node
W20260619 20:36:21.858975 26758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:21.859014 26756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:21.858870 26755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:21.859360 26750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:21.859599 26750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:21.860737 26750 hybrid_clock.cc:648] HybridClock initialized: now 1781901381860727 us; error 27 us; skew 500 ppm
Jun 19 20:36:21 dist-test-slave-0t40 krb5kdc[26375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901381, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:21.863313 26750 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:36:21.863770 26750 webserver.cc:460] Webserver started at http://127.25.62.131:32919/ using document root <none> and password file <none>
I20260619 20:36:21.863955 26750 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:21.863994 26750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:21.864080 26750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:21.865025 26750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "8e0adbf4383c4f6997a1ed13880cd216"
format_stamp: "Formatted at 2026-06-19 20:36:21 on dist-test-slave-0t40"
server_key: "bc3ab53fc0d38565f3397bd83cd3bf31"
server_key_iv: "8d8008721e174af41be2ae8c33a79eaa"
server_key_version: "encryptionkey@0"
I20260619 20:36:21.865397 26750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "8e0adbf4383c4f6997a1ed13880cd216"
format_stamp: "Formatted at 2026-06-19 20:36:21 on dist-test-slave-0t40"
server_key: "bc3ab53fc0d38565f3397bd83cd3bf31"
server_key_iv: "8d8008721e174af41be2ae8c33a79eaa"
server_key_version: "encryptionkey@0"
I20260619 20:36:21.866672 26750 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:21.867419 26765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:21.867686 26750 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:21.867792 26750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "8e0adbf4383c4f6997a1ed13880cd216"
format_stamp: "Formatted at 2026-06-19 20:36:21 on dist-test-slave-0t40"
server_key: "bc3ab53fc0d38565f3397bd83cd3bf31"
server_key_iv: "8d8008721e174af41be2ae8c33a79eaa"
server_key_version: "encryptionkey@0"
I20260619 20:36:21.867861 26750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:21.884215 26750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:21.886281 26750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:21.886442 26750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:21.886665 26750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:21.886971 26750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:21.887029 26750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:21.887069 26750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:21.887095 26750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:21.892639 26750 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:40593
I20260619 20:36:21.892740 26878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:40593 every 8 connection(s)
I20260619 20:36:21.893018 26750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:36:21.893533 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 26750
I20260619 20:36:21.893644 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:36:21.893839 25850 external_mini_cluster.cc:1428] Setting key 96109f15eaf9af4fd91351f216f9951b
Jun 19 20:36:21 dist-test-slave-0t40 krb5kdc[26375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901381, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:21.901860 26459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:21.893754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:47117 (local address 127.25.62.190:41785)
0619 20:36:21.893831 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:21.893833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:21.893980 (+ 147us) server_negotiation.cc:407] Connection header received
0619 20:36:21.895144 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:21.895145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:21.895154 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:21.895197 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:21.895750 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.896233 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.897184 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.897371 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.897440 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:21.899482 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:21.899483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:21.899483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:21.899489 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:21.900712 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:21.901101 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:21.901102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:21.901102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:21.901157 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:21.901371 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:21.901371 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:21.901372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:21.901475 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:21.901524 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:21.901744 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 20:36:21.901799 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 20:36:21.902010 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:21.893723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:41785 (local address 127.25.62.131:47117)
0619 20:36:21.893868 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:21.893908 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:21.895056 (+ 1148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:21.895254 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:21.895257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:21.895500 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:21.895682 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:21.895686 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.896312 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.896313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:21.897111 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:21.897113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.897453 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.897454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:21.897553 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:21.898065 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:21.898074 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:21.899402 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:21.900785 (+ 1383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:21.900790 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:21.900792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:21.900992 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:21.901219 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:21.901219 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:21.901220 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:21.901312 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:21.901532 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:21.901535 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:21.901639 (+ 104us) client_negotiation.cc:769] Sending connection context
0619 20:36:21.901715 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 20:36:21.901821 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260619 20:36:21.902488 26879 heartbeater.cc:347] Connected to a master server at 127.25.62.190:41785
I20260619 20:36:21.902570 26879 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:21.902693 26879 heartbeater.cc:511] Master 127.25.62.190:41785 requested a full tablet report, sending...
I20260619 20:36:21.903033 26420 ts_manager.cc:194] Registered new tserver with Master: 8e0adbf4383c4f6997a1ed13880cd216 (127.25.62.131:40593)
I20260619 20:36:21.903409 26420 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:47117
I20260619 20:36:21.905645 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:36:21.912758 26459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:21.907051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36502 (local address 127.25.62.190:41785)
0619 20:36:21.907142 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:21.907145 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:21.907256 (+ 111us) server_negotiation.cc:407] Connection header received
0619 20:36:21.907330 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:21.907331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:21.907340 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:21.907395 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:21.907952 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.908405 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.909171 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.909368 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.909461 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:21.910503 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:21.910504 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:21.910505 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:21.910511 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:21.911585 (+ 1074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:21.911966 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:21.911967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:21.911967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:21.912018 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:21.912226 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:21.912227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:21.912228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:21.912367 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:21.912491 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:21.912598 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 20:36:21.912665 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 20:36:21.914659 26420 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:36:21.916103 26420 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:36:21.922503 26887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:21.918160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:40593 (local address 127.0.0.1:44378)
0619 20:36:21.918319 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:21.918357 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:21.918421 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:21.918727 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:21.918730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:21.918732 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:21.918799 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:21.918801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.920105 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.920106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:21.921301 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:21.921303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.922193 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.922194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:21.922274 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:21.922278 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:21.922340 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 20:36:21.922393 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":49,"thread_start_us":61,"threads_started":1}
I20260619 20:36:21.922523 26891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:21.918415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46834 (local address 127.25.62.129:34221)
0619 20:36:21.918643 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:21.918646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:21.918656 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:36:21.918731 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:21.918733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:21.918748 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:21.918825 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:21.919034 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.920080 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.921489 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.922078 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.922178 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:21.922238 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:21.922397 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 20:36:21.922438 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":73,"threads_started":1}
I20260619 20:36:21.922513 26888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:21.918203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:34221 (local address 127.0.0.1:46834)
0619 20:36:21.918448 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:21.918458 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:21.918508 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:21.918857 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:21.918858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:21.918860 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:21.918911 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:21.918912 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.920169 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.920170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:21.921387 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:21.921390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.922181 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.922182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:21.922260 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:21.922273 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 20:36:21.922381 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 20:36:21.922425 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":29,"thread_start_us":59,"threads_started":1}
I20260619 20:36:21.922811 26889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:21.918233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44378 (local address 127.25.62.131:40593)
0619 20:36:21.918467 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:21.918474 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:21.918484 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:36:21.918560 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:21.918562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:21.918580 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:21.918664 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:21.918887 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.919994 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.921388 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.922038 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.922104 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:21.922158 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:21.922692 (+ 534us) server_negotiation.cc:299] Negotiation successful
0619 20:36:21.922731 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":80,"threads_started":1}
I20260619 20:36:21.923004 26892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:21.918602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42818 (local address 127.25.62.130:44359)
0619 20:36:21.918828 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:21.918832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:21.919038 (+ 206us) server_negotiation.cc:407] Connection header received
0619 20:36:21.919139 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:21.919141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:21.919218 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:21.919292 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:21.919513 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.920400 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.921685 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.922401 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.922479 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:21.922528 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:21.922892 (+ 364us) server_negotiation.cc:299] Negotiation successful
0619 20:36:21.922932 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":110,"threads_started":1}
I20260619 20:36:21.922770 26890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:21.918351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:44359 (local address 127.0.0.1:42818)
0619 20:36:21.918921 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:21.918931 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:21.918978 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:21.919394 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:21.919395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:21.919396 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:21.919431 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:21.919432 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.920515 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.920516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:21.921599 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:21.921601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:21.922561 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:21.922562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:21.922625 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:21.922629 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:21.922676 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 20:36:21.922713 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":41,"threads_started":1}
I20260619 20:36:21.923513 26813 tablet_service.cc:1467] Processing CreateTablet for tablet 4c1b6a128edb43b7a15ca7098a254d3a (DEFAULT_TABLE table=test-table [id=01f6cfb18eb941d5af07bec1c0a63681]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:21.923600 26545 tablet_service.cc:1467] Processing CreateTablet for tablet 4c1b6a128edb43b7a15ca7098a254d3a (DEFAULT_TABLE table=test-table [id=01f6cfb18eb941d5af07bec1c0a63681]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:21.923877 26813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c1b6a128edb43b7a15ca7098a254d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:21.923949 26545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c1b6a128edb43b7a15ca7098a254d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:21.925701 26679 tablet_service.cc:1467] Processing CreateTablet for tablet 4c1b6a128edb43b7a15ca7098a254d3a (DEFAULT_TABLE table=test-table [id=01f6cfb18eb941d5af07bec1c0a63681]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:21.926028 26679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c1b6a128edb43b7a15ca7098a254d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:21.926687 26893 tablet_bootstrap.cc:492] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726: Bootstrap starting.
I20260619 20:36:21.926980 26894 tablet_bootstrap.cc:492] T 4c1b6a128edb43b7a15ca7098a254d3a P 8e0adbf4383c4f6997a1ed13880cd216: Bootstrap starting.
I20260619 20:36:21.927404 26893 tablet_bootstrap.cc:654] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:21.927628 26894 tablet_bootstrap.cc:654] T 4c1b6a128edb43b7a15ca7098a254d3a P 8e0adbf4383c4f6997a1ed13880cd216: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:21.927950 26893 log.cc:824] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:21.928357 26894 log.cc:824] T 4c1b6a128edb43b7a15ca7098a254d3a P 8e0adbf4383c4f6997a1ed13880cd216: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:21.928712 26893 tablet_bootstrap.cc:492] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726: No bootstrap required, opened a new log
I20260619 20:36:21.928791 26893 ts_tablet_manager.cc:1397] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:36:21.929097 26894 tablet_bootstrap.cc:492] T 4c1b6a128edb43b7a15ca7098a254d3a P 8e0adbf4383c4f6997a1ed13880cd216: No bootstrap required, opened a new log
I20260619 20:36:21.929162 26894 ts_tablet_manager.cc:1397] T 4c1b6a128edb43b7a15ca7098a254d3a P 8e0adbf4383c4f6997a1ed13880cd216: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:36:21.929446 26897 tablet_bootstrap.cc:492] T 4c1b6a128edb43b7a15ca7098a254d3a P f5e5c4b5c2814d15857ffecdda66f275: Bootstrap starting.
I20260619 20:36:21.930059 26897 tablet_bootstrap.cc:654] T 4c1b6a128edb43b7a15ca7098a254d3a P f5e5c4b5c2814d15857ffecdda66f275: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:21.930526 26897 log.cc:824] T 4c1b6a128edb43b7a15ca7098a254d3a P f5e5c4b5c2814d15857ffecdda66f275: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:21.930824 26893 raft_consensus.cc:348] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0adbf4383c4f6997a1ed13880cd216" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40593 } } peers { permanent_uuid: "64ce00011a7144b480b51b4d1edef726" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34221 } } peers { permanent_uuid: "f5e5c4b5c2814d15857ffecdda66f275" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44359 } }
I20260619 20:36:21.931020 26893 raft_consensus.cc:374] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:21.931063 26893 raft_consensus.cc:729] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64ce00011a7144b480b51b4d1edef726, State: Initialized, Role: FOLLOWER
I20260619 20:36:21.931178 26893 consensus_queue.cc:260] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0adbf4383c4f6997a1ed13880cd216" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40593 } } peers { permanent_uuid: "64ce00011a7144b480b51b4d1edef726" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34221 } } peers { permanent_uuid: "f5e5c4b5c2814d15857ffecdda66f275" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44359 } }
I20260619 20:36:21.931116 26894 raft_consensus.cc:348] T 4c1b6a128edb43b7a15ca7098a254d3a P 8e0adbf4383c4f6997a1ed13880cd216 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0adbf4383c4f6997a1ed13880cd216" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40593 } } peers { permanent_uuid: "64ce00011a7144b480b51b4d1edef726" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34221 } } peers { permanent_uuid: "f5e5c4b5c2814d15857ffecdda66f275" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44359 } }
I20260619 20:36:21.931264 26894 raft_consensus.cc:374] T 4c1b6a128edb43b7a15ca7098a254d3a P 8e0adbf4383c4f6997a1ed13880cd216 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:21.931242 26897 tablet_bootstrap.cc:492] T 4c1b6a128edb43b7a15ca7098a254d3a P f5e5c4b5c2814d15857ffecdda66f275: No bootstrap required, opened a new log
I20260619 20:36:21.931303 26894 raft_consensus.cc:729] T 4c1b6a128edb43b7a15ca7098a254d3a P 8e0adbf4383c4f6997a1ed13880cd216 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e0adbf4383c4f6997a1ed13880cd216, State: Initialized, Role: FOLLOWER
I20260619 20:36:21.931331 26897 ts_tablet_manager.cc:1397] T 4c1b6a128edb43b7a15ca7098a254d3a P f5e5c4b5c2814d15857ffecdda66f275: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:36:21.931448 26893 ts_tablet_manager.cc:1428] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:36:21.931414 26894 consensus_queue.cc:260] T 4c1b6a128edb43b7a15ca7098a254d3a P 8e0adbf4383c4f6997a1ed13880cd216 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0adbf4383c4f6997a1ed13880cd216" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40593 } } peers { permanent_uuid: "64ce00011a7144b480b51b4d1edef726" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34221 } } peers { permanent_uuid: "f5e5c4b5c2814d15857ffecdda66f275" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44359 } }
I20260619 20:36:21.931594 26611 heartbeater.cc:503] Master 127.25.62.190:41785 was elected leader, sending a full tablet report...
I20260619 20:36:21.931682 26894 ts_tablet_manager.cc:1428] T 4c1b6a128edb43b7a15ca7098a254d3a P 8e0adbf4383c4f6997a1ed13880cd216: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 20:36:21.931838 26879 heartbeater.cc:503] Master 127.25.62.190:41785 was elected leader, sending a full tablet report...
I20260619 20:36:21.933290 26897 raft_consensus.cc:348] T 4c1b6a128edb43b7a15ca7098a254d3a P f5e5c4b5c2814d15857ffecdda66f275 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0adbf4383c4f6997a1ed13880cd216" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40593 } } peers { permanent_uuid: "64ce00011a7144b480b51b4d1edef726" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34221 } } peers { permanent_uuid: "f5e5c4b5c2814d15857ffecdda66f275" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44359 } }
I20260619 20:36:21.933454 26897 raft_consensus.cc:374] T 4c1b6a128edb43b7a15ca7098a254d3a P f5e5c4b5c2814d15857ffecdda66f275 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:21.933497 26897 raft_consensus.cc:729] T 4c1b6a128edb43b7a15ca7098a254d3a P f5e5c4b5c2814d15857ffecdda66f275 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5e5c4b5c2814d15857ffecdda66f275, State: Initialized, Role: FOLLOWER
I20260619 20:36:21.933606 26897 consensus_queue.cc:260] T 4c1b6a128edb43b7a15ca7098a254d3a P f5e5c4b5c2814d15857ffecdda66f275 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0adbf4383c4f6997a1ed13880cd216" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40593 } } peers { permanent_uuid: "64ce00011a7144b480b51b4d1edef726" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34221 } } peers { permanent_uuid: "f5e5c4b5c2814d15857ffecdda66f275" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44359 } }
I20260619 20:36:21.933879 26897 ts_tablet_manager.cc:1428] T 4c1b6a128edb43b7a15ca7098a254d3a P f5e5c4b5c2814d15857ffecdda66f275: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260619 20:36:21.933947 26745 heartbeater.cc:503] Master 127.25.62.190:41785 was elected leader, sending a full tablet report...
W20260619 20:36:21.987932 26746 tablet.cc:2369] T 4c1b6a128edb43b7a15ca7098a254d3a P f5e5c4b5c2814d15857ffecdda66f275: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:36:22.084636 26612 tablet.cc:2369] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:36:22.093115 26899 raft_consensus.cc:482] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:36:22.093209 26899 raft_consensus.cc:504] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0adbf4383c4f6997a1ed13880cd216" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40593 } } peers { permanent_uuid: "64ce00011a7144b480b51b4d1edef726" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34221 } } peers { permanent_uuid: "f5e5c4b5c2814d15857ffecdda66f275" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44359 } }
I20260619 20:36:22.093531 26899 leader_election.cc:288] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e0adbf4383c4f6997a1ed13880cd216 (127.25.62.131:40593), f5e5c4b5c2814d15857ffecdda66f275 (127.25.62.130:44359)
I20260619 20:36:22.096621 26902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:22.093721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:44359 (local address 127.25.62.129:33461)
0619 20:36:22.093834 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:22.093846 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:22.093903 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:22.094114 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:22.094116 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:22.094118 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:22.094203 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:22.094205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:22.094864 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:22.094866 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:22.095665 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:22.095667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:22.096338 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:22.096340 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:22.096414 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:22.096423 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 20:36:22.096465 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 20:36:22.096532 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":8,"thread_start_us":70,"threads_started":1}
I20260619 20:36:22.096618 26892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:22.093900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:33461 (local address 127.25.62.130:44359)
0619 20:36:22.093985 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:22.093987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:22.093997 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:36:22.094025 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:22.094026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:22.094037 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:22.094096 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:22.094277 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:22.094781 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:22.095774 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:22.096163 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:22.096220 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:22.096227 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:22.096500 (+ 273us) server_negotiation.cc:299] Negotiation successful
0619 20:36:22.096543 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 20:36:22.096658 26889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:22.093810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:41069 (local address 127.25.62.131:40593)
0619 20:36:22.093857 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:22.093859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:22.094011 (+ 152us) server_negotiation.cc:407] Connection header received
0619 20:36:22.094041 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:22.094043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:22.094050 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:22.094124 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:22.094338 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:22.094801 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:22.095801 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:22.096169 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:22.096357 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:22.096365 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:22.096546 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 20:36:22.096589 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:36:22.096645 26903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:22.093726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:40593 (local address 127.25.62.129:41069)
0619 20:36:22.093906 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:22.093916 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:22.093993 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:22.094164 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:22.094165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:22.094167 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:22.094258 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:22.094260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:22.094928 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:22.094929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:22.095681 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:22.095683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:22.096371 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:22.096372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:22.096433 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:22.096436 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:36:22.096519 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 20:36:22.096556 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":2,"thread_start_us":111,"threads_started":1}
I20260619 20:36:22.097033 26699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c1b6a128edb43b7a15ca7098a254d3a" candidate_uuid: "64ce00011a7144b480b51b4d1edef726" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5e5c4b5c2814d15857ffecdda66f275" is_pre_election: true
I20260619 20:36:22.097244 26699 raft_consensus.cc:2454] T 4c1b6a128edb43b7a15ca7098a254d3a P f5e5c4b5c2814d15857ffecdda66f275 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64ce00011a7144b480b51b4d1edef726 in term 0.
I20260619 20:36:22.097347 26833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c1b6a128edb43b7a15ca7098a254d3a" candidate_uuid: "64ce00011a7144b480b51b4d1edef726" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e0adbf4383c4f6997a1ed13880cd216" is_pre_election: true
I20260619 20:36:22.097491 26500 leader_election.cc:302] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64ce00011a7144b480b51b4d1edef726, f5e5c4b5c2814d15857ffecdda66f275; no voters:
I20260619 20:36:22.097517 26833 raft_consensus.cc:2454] T 4c1b6a128edb43b7a15ca7098a254d3a P 8e0adbf4383c4f6997a1ed13880cd216 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64ce00011a7144b480b51b4d1edef726 in term 0.
I20260619 20:36:22.097609 26899 raft_consensus.cc:2781] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:36:22.097688 26899 raft_consensus.cc:482] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:36:22.097714 26899 raft_consensus.cc:3037] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:22.098387 26899 raft_consensus.cc:504] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0adbf4383c4f6997a1ed13880cd216" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40593 } } peers { permanent_uuid: "64ce00011a7144b480b51b4d1edef726" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34221 } } peers { permanent_uuid: "f5e5c4b5c2814d15857ffecdda66f275" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44359 } }
I20260619 20:36:22.098541 26899 leader_election.cc:288] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726 [CANDIDATE]: Term 1 election: Requested vote from peers 8e0adbf4383c4f6997a1ed13880cd216 (127.25.62.131:40593), f5e5c4b5c2814d15857ffecdda66f275 (127.25.62.130:44359)
I20260619 20:36:22.098655 26833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c1b6a128edb43b7a15ca7098a254d3a" candidate_uuid: "64ce00011a7144b480b51b4d1edef726" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e0adbf4383c4f6997a1ed13880cd216"
I20260619 20:36:22.098735 26833 raft_consensus.cc:3037] T 4c1b6a128edb43b7a15ca7098a254d3a P 8e0adbf4383c4f6997a1ed13880cd216 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:22.099354 26833 raft_consensus.cc:2454] T 4c1b6a128edb43b7a15ca7098a254d3a P 8e0adbf4383c4f6997a1ed13880cd216 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64ce00011a7144b480b51b4d1edef726 in term 1.
I20260619 20:36:22.099483 26699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c1b6a128edb43b7a15ca7098a254d3a" candidate_uuid: "64ce00011a7144b480b51b4d1edef726" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5e5c4b5c2814d15857ffecdda66f275"
I20260619 20:36:22.099534 26499 leader_election.cc:302] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64ce00011a7144b480b51b4d1edef726, 8e0adbf4383c4f6997a1ed13880cd216; no voters:
I20260619 20:36:22.099558 26699 raft_consensus.cc:3037] T 4c1b6a128edb43b7a15ca7098a254d3a P f5e5c4b5c2814d15857ffecdda66f275 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:22.099642 26899 raft_consensus.cc:2781] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:22.099776 26899 raft_consensus.cc:686] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726 [term 1 LEADER]: Becoming Leader. State: Replica: 64ce00011a7144b480b51b4d1edef726, State: Running, Role: LEADER
I20260619 20:36:22.099870 26899 consensus_queue.cc:237] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0adbf4383c4f6997a1ed13880cd216" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40593 } } peers { permanent_uuid: "64ce00011a7144b480b51b4d1edef726" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34221 } } peers { permanent_uuid: "f5e5c4b5c2814d15857ffecdda66f275" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44359 } }
I20260619 20:36:22.100468 26699 raft_consensus.cc:2454] T 4c1b6a128edb43b7a15ca7098a254d3a P f5e5c4b5c2814d15857ffecdda66f275 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64ce00011a7144b480b51b4d1edef726 in term 1.
I20260619 20:36:22.100674 26420 catalog_manager.cc:5495] T 4c1b6a128edb43b7a15ca7098a254d3a P 64ce00011a7144b480b51b4d1edef726 reported cstate change: term changed from 0 to 1, leader changed from <none> to 64ce00011a7144b480b51b4d1edef726 (127.25.62.129). New cstate: current_term: 1 leader_uuid: "64ce00011a7144b480b51b4d1edef726" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e0adbf4383c4f6997a1ed13880cd216" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64ce00011a7144b480b51b4d1edef726" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5e5c4b5c2814d15857ffecdda66f275" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44359 } health_report { overall_health: UNKNOWN } } }
Jun 19 20:36:22 dist-test-slave-0t40 krb5kdc[26375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.129@KRBTEST.COM
I20260619 20:36:22.110121 26891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:22.102802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46848 (local address 127.25.62.129:34221)
0619 20:36:22.102844 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:22.102847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:22.102957 (+ 110us) server_negotiation.cc:407] Connection header received
0619 20:36:22.102979 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:22.102980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:22.102988 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:22.103039 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:22.103500 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:22.103935 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:22.104628 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:22.104807 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:22.104869 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:22.107480 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:22.107489 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:22.107489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:22.107498 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:22.108856 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:22.109283 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:22.109287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:22.109287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:22.109364 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:22.109591 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:22.109592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:22.109592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:22.109726 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:22.109894 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:22.109932 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 20:36:22.110011 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Jun 19 20:36:22 dist-test-slave-0t40 krb5kdc[26375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901382, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 20:36:22 dist-test-slave-0t40 krb5kdc[26375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901382, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.62.129@KRBTEST.COM
I20260619 20:36:22.132221 26891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:22.125215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46852 (local address 127.25.62.129:34221)
0619 20:36:22.125292 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:22.125294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:22.125326 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:36:22.125402 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:22.125403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:22.125411 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:22.125461 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:22.125989 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:22.126450 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:22.127118 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:22.127294 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:22.127344 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:22.129707 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:22.129708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:22.129708 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:22.129715 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:22.131103 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:22.131432 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:22.131433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:22.131433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:22.131482 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:22.131654 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:22.131654 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:22.131654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:22.131741 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:22.131830 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:22.132044 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 20:36:22.132105 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Jun 19 20:36:22 dist-test-slave-0t40 krb5kdc[26375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260619 20:36:22.143610 26880 tablet.cc:2369] T 4c1b6a128edb43b7a15ca7098a254d3a P 8e0adbf4383c4f6997a1ed13880cd216: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 20:36:22 dist-test-slave-0t40 krb5kdc[26375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.129@KRBTEST.COM
I20260619 20:36:22.155143 26891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:22.147798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46860 (local address 127.25.62.129:34221)
0619 20:36:22.147878 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:22.147880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:22.147889 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:36:22.147940 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:22.147940 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:22.147949 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:22.148012 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:22.148560 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:22.149101 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:22.149900 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:22.150106 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:22.150162 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:22.152464 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:22.152465 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:22.152465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:22.152472 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:22.154006 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:22.154358 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:22.154359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:22.154359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:22.154410 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:22.154570 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:22.154571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:22.154571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:22.154670 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:22.154790 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:22.154949 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 20:36:22.155013 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260619 20:36:22.155599 26525 tablet_service.cc:2908] Rejecting scan request for tablet 4c1b6a128edb43b7a15ca7098a254d3a: Uninitialized: clean time has not yet been initialized
I20260619 20:36:22.156508 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 26482
I20260619 20:36:22.162153 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 26616
I20260619 20:36:22.167456 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 26750
I20260619 20:36:22.171844 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 26388
2026-06-19T20:36:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2072 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:22 dist-test-slave-0t40 krb5kdc[26918](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:22 dist-test-slave-0t40 krb5kdc[26918](info): set up 2 sockets
Jun 19 20:36:22 dist-test-slave-0t40 krb5kdc[26918](info): commencing operation
krb5kdc: starting...
W20260619 20:36:23.217285 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:36:23 dist-test-slave-0t40 krb5kdc[26918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:36:23.336242 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:36709
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:38543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:36709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:23.413957 26931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:23.414201 26931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:23.414237 26931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:23.415383 26931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:23.415460 26931 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 20:36:23.415484 26931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:23.415506 26931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:23.415535 26931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:23.415557 26931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:23.416996 26931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:38543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:36709
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:36709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:23.417286 26931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:23.417510 26931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:36:23.422416 26931 server_base.cc:1034] running on GCE node
W20260619 20:36:23.422344 26937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:23.422434 26939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:23.422344 26936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:23.422827 26931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:23.423130 26931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:23.424310 26931 hybrid_clock.cc:648] HybridClock initialized: now 1781901383424287 us; error 38 us; skew 500 ppm
Jun 19 20:36:23 dist-test-slave-0t40 krb5kdc[26918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901383, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:23.430226 26931 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:36:23.430673 26931 webserver.cc:460] Webserver started at http://127.25.62.190:44003/ using document root <none> and password file <none>
I20260619 20:36:23.430944 26931 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:23.431021 26931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:23.431181 26931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:23.432333 26931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "c7c352c4b87f45558b0c408b35351456"
format_stamp: "Formatted at 2026-06-19 20:36:23 on dist-test-slave-0t40"
server_key: "dfe3a78d93a7f0ecf56782c1f6a30094"
server_key_iv: "9ca0c9870765f4a965b0d0a2d20dbb4f"
server_key_version: "encryptionkey@0"
I20260619 20:36:23.432722 26931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "c7c352c4b87f45558b0c408b35351456"
format_stamp: "Formatted at 2026-06-19 20:36:23 on dist-test-slave-0t40"
server_key: "dfe3a78d93a7f0ecf56782c1f6a30094"
server_key_iv: "9ca0c9870765f4a965b0d0a2d20dbb4f"
server_key_version: "encryptionkey@0"
I20260619 20:36:23.434165 26931 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:36:23.435375 26946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:23.435607 26931 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:23.435705 26931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "c7c352c4b87f45558b0c408b35351456"
format_stamp: "Formatted at 2026-06-19 20:36:23 on dist-test-slave-0t40"
server_key: "dfe3a78d93a7f0ecf56782c1f6a30094"
server_key_iv: "9ca0c9870765f4a965b0d0a2d20dbb4f"
server_key_version: "encryptionkey@0"
I20260619 20:36:23.435791 26931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:23.445626 26931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:23.447326 26931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:23.447494 26931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:23.452234 26931 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:36709
I20260619 20:36:23.452283 27008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:36709 every 8 connection(s)
I20260619 20:36:23.452584 26931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:23.453212 27009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:23.455413 27009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456: Bootstrap starting.
I20260619 20:36:23.455998 27009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:23.456281 27009 log.cc:824] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:23.456943 27009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456: No bootstrap required, opened a new log
I20260619 20:36:23.459079 27009 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c352c4b87f45558b0c408b35351456" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 36709 } }
I20260619 20:36:23.459244 27009 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:23.459290 27009 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7c352c4b87f45558b0c408b35351456, State: Initialized, Role: FOLLOWER
I20260619 20:36:23.459430 27009 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c352c4b87f45558b0c408b35351456" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 36709 } }
I20260619 20:36:23.459515 27009 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:23.459558 27009 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:23.459604 27009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:23.460119 27009 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c352c4b87f45558b0c408b35351456" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 36709 } }
I20260619 20:36:23.460235 27009 leader_election.cc:302] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7c352c4b87f45558b0c408b35351456; no voters:
I20260619 20:36:23.460417 27009 leader_election.cc:288] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:23.460491 27012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:23.460534 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 26931
I20260619 20:36:23.460615 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:36:23.460685 27009 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:23.460712 27012 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456 [term 1 LEADER]: Becoming Leader. State: Replica: c7c352c4b87f45558b0c408b35351456, State: Running, Role: LEADER
I20260619 20:36:23.460783 25850 external_mini_cluster.cc:1428] Setting key f5c98da7b98ddac6df4da8ebdc892abe
I20260619 20:36:23.460870 27012 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c352c4b87f45558b0c408b35351456" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 36709 } }
I20260619 20:36:23.461283 27013 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7c352c4b87f45558b0c408b35351456" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c352c4b87f45558b0c408b35351456" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 36709 } } }
I20260619 20:36:23.461372 27013 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:23.461464 27014 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7c352c4b87f45558b0c408b35351456. Latest consensus state: current_term: 1 leader_uuid: "c7c352c4b87f45558b0c408b35351456" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7c352c4b87f45558b0c408b35351456" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 36709 } } }
I20260619 20:36:23.461541 27014 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:23.461665 27022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:23.462199 27022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:23.463776 27022 catalog_manager.cc:1347] Generated new cluster ID: cec6027650ca4c0f99d6e2160e0367a1
I20260619 20:36:23.463828 27022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:36:23 dist-test-slave-0t40 krb5kdc[26918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:23.473006 27019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.461272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45286 (local address 127.25.62.190:36709)
0619 20:36:23.462105 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:23.462115 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:23.462147 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:36:23.462263 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:23.462270 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:23.462404 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:23.462586 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:23.463330 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.464755 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.465881 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.466485 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.466580 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.469406 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:23.469410 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:23.469410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:23.469423 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:23.471385 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:23.471770 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:23.471773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:23.471773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:23.471891 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:23.472098 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:23.472099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:23.472100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:23.472415 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:23.472565 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:23.472638 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 20:36:23.472767 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":788,"thread_start_us":73,"threads_started":1}
I20260619 20:36:23.480223 27022 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:23.480955 27022 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:23.488624 27022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456: Generated new TSK 0
I20260619 20:36:23.488811 27022 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
I20260619 20:36:23.519932 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:36709
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:38543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:23.602463 27035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:23.602699 27035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:23.602741 27035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:23.603874 27035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:23.603956 27035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:23.603992 27035 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:36:23.604035 27035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:36:23.605468 27035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:38543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.62.190:36709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:23.605731 27035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:23.605980 27035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:23.611037 27040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:23.611037 27041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:23.611042 27043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:23.611167 27035 server_base.cc:1034] running on GCE node
I20260619 20:36:23.611444 27035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:23.611701 27035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:23.612867 27035 hybrid_clock.cc:648] HybridClock initialized: now 1781901383612858 us; error 38 us; skew 500 ppm
Jun 19 20:36:23 dist-test-slave-0t40 krb5kdc[26918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901383, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:23.615388 27035 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:36:23.615818 27035 webserver.cc:460] Webserver started at http://127.25.62.129:32901/ using document root <none> and password file <none>
I20260619 20:36:23.616003 27035 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:23.616041 27035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:23.616133 27035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:23.617136 27035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "b9952969eb524d9bb6ecfd0758cedba4"
format_stamp: "Formatted at 2026-06-19 20:36:23 on dist-test-slave-0t40"
server_key: "6a4bfeb85574d747f21b4cabb4280c99"
server_key_iv: "6f81c73e38a8d4eef48b795a300a391a"
server_key_version: "encryptionkey@0"
I20260619 20:36:23.617440 27035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "b9952969eb524d9bb6ecfd0758cedba4"
format_stamp: "Formatted at 2026-06-19 20:36:23 on dist-test-slave-0t40"
server_key: "6a4bfeb85574d747f21b4cabb4280c99"
server_key_iv: "6f81c73e38a8d4eef48b795a300a391a"
server_key_version: "encryptionkey@0"
I20260619 20:36:23.618666 27035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:23.619458 27050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:23.619663 27035 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:23.619755 27035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "b9952969eb524d9bb6ecfd0758cedba4"
format_stamp: "Formatted at 2026-06-19 20:36:23 on dist-test-slave-0t40"
server_key: "6a4bfeb85574d747f21b4cabb4280c99"
server_key_iv: "6f81c73e38a8d4eef48b795a300a391a"
server_key_version: "encryptionkey@0"
I20260619 20:36:23.619843 27035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:23.629506 27035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:23.631680 27035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:23.631856 27035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:23.632395 27035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:23.632458 27035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:23.632501 27035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:23.632529 27035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 20:36:23 dist-test-slave-0t40 krb5kdc[26918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901383, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:23.639825 27035 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:36935
I20260619 20:36:23.640172 27035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:36:23.639950 27165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:36935 every 8 connection(s)
I20260619 20:36:23.641809 27019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.632941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:55965 (local address 127.25.62.190:36709)
0619 20:36:23.632983 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:23.632988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:23.633238 (+ 250us) server_negotiation.cc:407] Connection header received
0619 20:36:23.633727 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:23.633728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:23.633743 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:23.633806 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:23.634383 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.634959 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.635929 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.636184 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.636267 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.638836 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:23.638838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:23.638838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:23.638846 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:23.640660 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:23.641146 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:23.641147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:23.641147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:23.641197 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:23.641370 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:23.641370 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:23.641370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:23.641466 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:23.641510 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:23.641690 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 20:36:23.641749 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:36:23.641954 27059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.632914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:36709 (local address 127.25.62.129:55965)
0619 20:36:23.633111 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:23.633157 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:23.633620 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:23.633867 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:23.633871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:23.634105 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:23.634275 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:23.634278 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.635052 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.635053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:23.635816 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:23.635817 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.636279 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.636280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:23.636400 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.636885 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:23.636894 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:23.638732 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:23.640722 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:23.640725 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:23.640726 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:23.641033 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:23.641242 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:23.641243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:23.641243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:23.641332 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:23.641500 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:23.641502 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:23.641598 (+ 96us) client_negotiation.cc:769] Sending connection context
0619 20:36:23.641669 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 20:36:23.641775 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":89,"threads_started":1}
I20260619 20:36:23.642488 27166 heartbeater.cc:347] Connected to a master server at 127.25.62.190:36709
I20260619 20:36:23.642565 27166 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:23.642732 27166 heartbeater.cc:511] Master 127.25.62.190:36709 requested a full tablet report, sending...
I20260619 20:36:23.643194 26963 ts_manager.cc:194] Registered new tserver with Master: b9952969eb524d9bb6ecfd0758cedba4 (127.25.62.129:36935)
I20260619 20:36:23.644016 26963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:55965
I20260619 20:36:23.644958 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 27035
I20260619 20:36:23.645071 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:36:23.645239 25850 external_mini_cluster.cc:1428] Setting key 4061d4927f5efd6dd83166819e0226b3
I20260619 20:36:23.649160 27019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.643504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:59463 (local address 127.25.62.190:36709)
0619 20:36:23.643545 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:23.643547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:23.643617 (+ 70us) server_negotiation.cc:407] Connection header received
0619 20:36:23.643670 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:23.643671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:23.643679 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:23.643741 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:23.644003 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.644515 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.645435 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.645725 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.645787 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.646697 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:23.646698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:23.646699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:23.646705 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:23.648031 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:23.648391 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:23.648392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:23.648393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:23.648444 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:23.648656 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:23.648657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:23.648657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:23.648763 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:23.648839 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:23.648983 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 20:36:23.649078 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:36:23.649201 27172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.643445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:36709 (local address 127.25.62.129:59463)
0619 20:36:23.643560 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:23.643570 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:23.643630 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:23.643744 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:23.643744 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:23.643894 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:23.643938 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:23.643940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.644622 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.644623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:23.645353 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:23.645355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.645801 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.645802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:23.645882 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.646318 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:23.646323 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:23.646583 (+ 260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:23.648141 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:23.648142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:23.648142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:23.648303 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:23.648521 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:23.648522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:23.648523 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:23.648577 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:23.648832 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:23.648833 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:23.648873 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 20:36:23.648960 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 20:36:23.649074 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.25.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
I20260619 20:36:23.669870 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:36709
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:38543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:23.748522 27175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:23.748713 27175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:23.748736 27175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:23.749825 27175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:23.749856 27175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:23.749876 27175 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:36:23.749893 27175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:36:23.751204 27175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:38543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.62.190:36709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:23.751380 27175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:23.751539 27175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:23.756484 27183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:23.756460 27180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:23.756475 27181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:23.756588 27175 server_base.cc:1034] running on GCE node
I20260619 20:36:23.756867 27175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:23.757179 27175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:23.758374 27175 hybrid_clock.cc:648] HybridClock initialized: now 1781901383758363 us; error 31 us; skew 500 ppm
Jun 19 20:36:23 dist-test-slave-0t40 krb5kdc[26918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901383, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:23.760952 27175 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:36:23.761529 27175 webserver.cc:460] Webserver started at http://127.25.62.130:33713/ using document root <none> and password file <none>
I20260619 20:36:23.761750 27175 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:23.761792 27175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:23.761905 27175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:23.762966 27175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b"
format_stamp: "Formatted at 2026-06-19 20:36:23 on dist-test-slave-0t40"
server_key: "654fbe7a5062ece62ffda3803119e3c4"
server_key_iv: "9fd4d3f5b5d90228c0e28a2d7d604b8d"
server_key_version: "encryptionkey@0"
I20260619 20:36:23.763255 27175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b"
format_stamp: "Formatted at 2026-06-19 20:36:23 on dist-test-slave-0t40"
server_key: "654fbe7a5062ece62ffda3803119e3c4"
server_key_iv: "9fd4d3f5b5d90228c0e28a2d7d604b8d"
server_key_version: "encryptionkey@0"
I20260619 20:36:23.764569 27175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:36:23.765354 27190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:23.765532 27175 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:23.765612 27175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b"
format_stamp: "Formatted at 2026-06-19 20:36:23 on dist-test-slave-0t40"
server_key: "654fbe7a5062ece62ffda3803119e3c4"
server_key_iv: "9fd4d3f5b5d90228c0e28a2d7d604b8d"
server_key_version: "encryptionkey@0"
I20260619 20:36:23.765663 27175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:23.774443 27175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:23.776611 27175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:23.776778 27175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:23.777539 27175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:23.777582 27175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:23.777606 27175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:23.777619 27175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:23.784399 27175 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:35913
Jun 19 20:36:23 dist-test-slave-0t40 krb5kdc[26918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901383, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:23.784548 27309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:35913 every 8 connection(s)
I20260619 20:36:23.784806 27175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:36:23.787546 27019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.777984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:50353 (local address 127.25.62.190:36709)
0619 20:36:23.778039 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:23.778042 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:23.778268 (+ 226us) server_negotiation.cc:407] Connection header received
0619 20:36:23.778708 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:23.778710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:23.778725 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:23.778818 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:23.779488 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.780329 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.781918 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.782265 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.782347 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.785069 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:23.785070 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:23.785071 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:23.785079 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:23.786502 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:23.786841 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:23.786841 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:23.786842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:23.786888 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:23.787090 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:23.787091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:23.787091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:23.787185 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:23.787233 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:23.787432 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 20:36:23.787486 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:36:23.787729 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.777972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:36709 (local address 127.25.62.130:50353)
0619 20:36:23.778120 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:23.778159 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:23.778610 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:23.778864 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:23.778867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:23.779088 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:23.779373 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:23.779380 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.780426 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.780427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:23.781822 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:23.781825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.782378 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.782379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:23.782526 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.783087 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:23.783099 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:23.784942 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:23.786572 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:23.786576 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:23.786577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:23.786776 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:23.786945 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:23.786946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:23.786946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:23.787034 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:23.787239 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:23.787243 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:23.787344 (+ 101us) client_negotiation.cc:769] Sending connection context
0619 20:36:23.787421 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 20:36:23.787539 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260619 20:36:23.788196 27310 heartbeater.cc:347] Connected to a master server at 127.25.62.190:36709
I20260619 20:36:23.788285 27310 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:23.788450 27310 heartbeater.cc:511] Master 127.25.62.190:36709 requested a full tablet report, sending...
I20260619 20:36:23.788820 26963 ts_manager.cc:194] Registered new tserver with Master: 6e5e0e4edf7e4dc2bd9d9b04050c357b (127.25.62.130:35913)
I20260619 20:36:23.789304 26963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:50353
I20260619 20:36:23.794148 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 27175
I20260619 20:36:23.794250 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:36:23.794314 27019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.789272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:41297 (local address 127.25.62.190:36709)
0619 20:36:23.789349 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:23.789351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:23.789409 (+ 58us) server_negotiation.cc:407] Connection header received
0619 20:36:23.789480 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:23.789481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:23.789490 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:23.789542 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:23.789854 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.790307 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.790994 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.791182 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.791256 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.792022 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:23.792024 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:23.792024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:23.792030 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:23.793173 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:23.793513 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:23.793514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:23.793515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:23.793577 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:23.793775 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:23.793776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:23.793776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:23.793918 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:23.793994 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:23.794161 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 20:36:23.794231 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 20:36:23.794333 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.789214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:36709 (local address 127.25.62.130:41297)
0619 20:36:23.789349 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:23.789360 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:23.789431 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:23.789558 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:23.789558 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:23.789722 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:23.789776 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:23.789777 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.790402 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.790403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:23.790901 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:23.790903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.791255 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.791256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:23.791326 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.791681 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:23.791684 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:23.791925 (+ 241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:23.793261 (+ 1336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:23.793262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:23.793263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:23.793427 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:23.793644 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:23.793645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:23.793645 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:23.793702 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:23.794001 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:23.794001 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:23.794044 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 20:36:23.794136 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 20:36:23.794233 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":84,"threads_started":1}
I20260619 20:36:23.794473 25850 external_mini_cluster.cc:1428] Setting key 4f6594507a48c6cc05d789aa1b33c9ee
WARNING: no policy specified for kudu/127.25.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
I20260619 20:36:23.820187 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:36709
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:38543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:23.900511 27320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:23.900709 27320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:23.900733 27320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:23.901969 27320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:23.902014 27320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:23.902034 27320 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:36:23.902052 27320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:36:23.903929 27320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:38543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.62.190:36709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:23.904170 27320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:23.904392 27320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:36:23.909634 27320 server_base.cc:1034] running on GCE node
W20260619 20:36:23.909565 27326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:23.909557 27325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:23.909564 27329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:23.910034 27320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:23.910311 27320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:23.911510 27320 hybrid_clock.cc:648] HybridClock initialized: now 1781901383911504 us; error 39 us; skew 500 ppm
Jun 19 20:36:23 dist-test-slave-0t40 krb5kdc[26918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901383, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:23.914244 27320 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:36:23.914752 27320 webserver.cc:460] Webserver started at http://127.25.62.131:38173/ using document root <none> and password file <none>
I20260619 20:36:23.914943 27320 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:23.914992 27320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:23.915082 27320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:23.916091 27320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "9462e18558364f33830ed4ae90a9c987"
format_stamp: "Formatted at 2026-06-19 20:36:23 on dist-test-slave-0t40"
server_key: "bc9b4d52f1372bcabfe6d46c4e6c1abd"
server_key_iv: "259a9c045df315441a620abb883b302b"
server_key_version: "encryptionkey@0"
I20260619 20:36:23.916391 27320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "9462e18558364f33830ed4ae90a9c987"
format_stamp: "Formatted at 2026-06-19 20:36:23 on dist-test-slave-0t40"
server_key: "bc9b4d52f1372bcabfe6d46c4e6c1abd"
server_key_iv: "259a9c045df315441a620abb883b302b"
server_key_version: "encryptionkey@0"
I20260619 20:36:23.917821 27320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:36:23.918625 27336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:23.918823 27320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:23.918932 27320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "9462e18558364f33830ed4ae90a9c987"
format_stamp: "Formatted at 2026-06-19 20:36:23 on dist-test-slave-0t40"
server_key: "bc9b4d52f1372bcabfe6d46c4e6c1abd"
server_key_iv: "259a9c045df315441a620abb883b302b"
server_key_version: "encryptionkey@0"
I20260619 20:36:23.918998 27320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:23.931794 27320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:23.934432 27320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:23.934630 27320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:23.935168 27320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:23.935213 27320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:23.935241 27320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:23.935303 27320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:23.943456 27320 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:43469
I20260619 20:36:23.943511 27456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:43469 every 8 connection(s)
Jun 19 20:36:23 dist-test-slave-0t40 krb5kdc[26918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901383, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:23.943922 27320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:36:23.944785 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 27320
I20260619 20:36:23.944875 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:36:23.945080 25850 external_mini_cluster.cc:1428] Setting key 96b16778db1d01e095ccfe4664463097
I20260619 20:36:23.947481 27019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.935749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:44557 (local address 127.25.62.190:36709)
0619 20:36:23.935828 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:23.935831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:23.936058 (+ 227us) server_negotiation.cc:407] Connection header received
0619 20:36:23.936586 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:23.936587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:23.936600 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:23.936664 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:23.937697 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.938554 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.940299 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.940626 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.940704 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.944317 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:23.944319 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:23.944320 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:23.944328 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:23.946070 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:23.946502 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:23.946503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:23.946504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:23.946559 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:23.946823 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:23.946824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:23.946824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:23.946950 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:23.947027 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:23.947308 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 20:36:23.947387 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 20:36:23.947691 27345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.935723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:36709 (local address 127.25.62.131:44557)
0619 20:36:23.935902 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:23.935953 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:23.936459 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:23.936792 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:23.936798 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:23.937231 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:23.937573 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:23.937581 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.938668 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.938670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:23.940197 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:23.940201 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.940745 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.940746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:23.940909 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.942106 (+ 1197us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:23.942117 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:23.944172 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:23.946166 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:23.946173 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:23.946175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:23.946400 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:23.946636 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:23.946637 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:23.946637 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:23.946751 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:23.947031 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:23.947037 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:23.947175 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 20:36:23.947287 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 20:36:23.947456 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260619 20:36:23.948228 27457 heartbeater.cc:347] Connected to a master server at 127.25.62.190:36709
I20260619 20:36:23.948472 27457 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:23.948727 27457 heartbeater.cc:511] Master 127.25.62.190:36709 requested a full tablet report, sending...
I20260619 20:36:23.949205 26962 ts_manager.cc:194] Registered new tserver with Master: 9462e18558364f33830ed4ae90a9c987 (127.25.62.131:43469)
I20260619 20:36:23.949725 26962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:44557
I20260619 20:36:23.954514 27463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.949236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:36709 (local address 127.25.62.131:35255)
0619 20:36:23.949404 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:23.949412 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:23.949461 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:23.949593 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:23.949593 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:23.949744 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:23.949792 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:23.949793 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.950451 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.950452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:23.950971 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:23.950973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.951363 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.951364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:23.951439 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.951810 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:23.951814 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:23.952079 (+ 265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:23.953486 (+ 1407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:23.953487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:23.953487 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:23.953653 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:23.953868 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:23.953869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:23.953870 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:23.953925 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:23.954189 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:23.954190 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:23.954233 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 20:36:23.954318 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 20:36:23.954413 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":64,"threads_started":1}
I20260619 20:36:23.954506 27019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.949301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:35255 (local address 127.25.62.190:36709)
0619 20:36:23.949338 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:23.949340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:23.949481 (+ 141us) server_negotiation.cc:407] Connection header received
0619 20:36:23.949506 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:23.949507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:23.949518 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:23.949592 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:23.949892 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.950353 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.951072 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.951276 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.951361 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.952170 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:23.952171 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:23.952172 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:23.952178 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:23.953378 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:23.953741 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:23.953742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:23.953742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:23.953799 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:23.954015 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:23.954016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:23.954016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:23.954121 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:23.954197 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:23.954341 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 20:36:23.954411 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 20:36:23.956928 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 20:36:23 dist-test-slave-0t40 krb5kdc[26918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901383, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 20:36:23 dist-test-slave-0t40 krb5kdc[26918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901383, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:23.979205 27019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.972266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45288 (local address 127.25.62.190:36709)
0619 20:36:23.972328 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:23.972330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:23.972342 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:36:23.972417 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:23.972418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:23.972427 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:23.972481 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:23.973079 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.973605 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.974292 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.974462 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.974511 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.976557 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:23.976558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:23.976559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:23.976565 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:23.977991 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:23.978355 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:23.978356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:23.978357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:23.978415 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:23.978616 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:23.978617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:23.978618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:23.978739 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:23.978825 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:23.978994 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 20:36:23.979059 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 20:36:23.980841 26962 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:36:23.982503 26962 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:36:23.989457 27472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.984898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:36935 (local address 127.0.0.1:54636)
0619 20:36:23.985199 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:23.985243 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:23.985305 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:23.985859 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:23.985860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:23.985863 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:23.985923 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:23.985925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.986854 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.986855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:23.988229 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:23.988233 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.989225 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.989226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:23.989279 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.989282 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:36:23.989332 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 20:36:23.989364 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":86,"thread_start_us":72,"threads_started":1}
I20260619 20:36:23.989557 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.985022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54636 (local address 127.25.62.129:36935)
0619 20:36:23.985513 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:23.985517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:23.985527 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:36:23.985691 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:23.985694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:23.985709 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:23.985784 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:23.986031 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.986735 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.988328 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.989017 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.989100 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.989177 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:23.989376 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 20:36:23.989436 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":115,"threads_started":1}
I20260619 20:36:23.989676 27476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.985115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45198 (local address 127.25.62.130:35913)
0619 20:36:23.985403 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:23.985408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:23.985419 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:36:23.985462 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:23.985464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:23.985481 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:23.985557 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:23.985708 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.986853 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.988380 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.988887 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.989008 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.989103 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:23.989520 (+ 417us) server_negotiation.cc:299] Negotiation successful
0619 20:36:23.989565 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":77,"threads_started":1}
I20260619 20:36:23.989456 27475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.985104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:43469 (local address 127.0.0.1:42460)
0619 20:36:23.985430 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:23.985439 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:23.985486 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:23.985762 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:23.985763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:23.985766 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:23.985806 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:23.985808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.987055 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.987055 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:23.988085 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:23.988087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.989153 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.989154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:23.989230 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.989242 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 20:36:23.989302 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 20:36:23.989354 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":34,"threads_started":1}
I20260619 20:36:23.989895 27473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.984898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:35913 (local address 127.0.0.1:45198)
0619 20:36:23.985308 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:23.985322 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:23.985384 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:23.985562 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:23.985564 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:23.985566 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:23.985628 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:23.985630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.986921 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.986921 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:23.988269 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:23.988271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.989251 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.989252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:23.989336 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.989341 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:36:23.989407 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 20:36:23.989448 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":71,"thread_start_us":55,"threads_started":1}
I20260619 20:36:23.990226 27477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:23.985193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42460 (local address 127.25.62.131:43469)
0619 20:36:23.985397 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:23.985401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:23.985479 (+ 78us) server_negotiation.cc:407] Connection header received
0619 20:36:23.985566 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:23.985568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:23.985585 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:23.985672 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:23.985885 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.986960 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.988229 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:23.988946 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:23.989017 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:23.990018 (+ 1001us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:23.990050 (+ 32us) server_negotiation.cc:299] Negotiation successful
0619 20:36:23.990109 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":87,"threads_started":1}
I20260619 20:36:23.990216 27100 tablet_service.cc:1467] Processing CreateTablet for tablet d3e0e2a871ea476683c2fb7e05e8cc6c (DEFAULT_TABLE table=test-table [id=98edb63443a349598ab5b3e435a5648b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:23.990466 27240 tablet_service.cc:1467] Processing CreateTablet for tablet d3e0e2a871ea476683c2fb7e05e8cc6c (DEFAULT_TABLE table=test-table [id=98edb63443a349598ab5b3e435a5648b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:23.990595 27100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e0e2a871ea476683c2fb7e05e8cc6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:23.990774 27240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e0e2a871ea476683c2fb7e05e8cc6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:23.990793 27389 tablet_service.cc:1467] Processing CreateTablet for tablet d3e0e2a871ea476683c2fb7e05e8cc6c (DEFAULT_TABLE table=test-table [id=98edb63443a349598ab5b3e435a5648b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:23.991160 27389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e0e2a871ea476683c2fb7e05e8cc6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:23.993796 27478 tablet_bootstrap.cc:492] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987: Bootstrap starting.
I20260619 20:36:23.994436 27479 tablet_bootstrap.cc:492] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Bootstrap starting.
I20260619 20:36:23.994683 27480 tablet_bootstrap.cc:492] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4: Bootstrap starting.
I20260619 20:36:23.994798 27478 tablet_bootstrap.cc:654] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:23.995085 27479 tablet_bootstrap.cc:654] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:23.995162 27478 log.cc:824] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:23.995297 27480 tablet_bootstrap.cc:654] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:23.995435 27479 log.cc:824] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:23.995731 27480 log.cc:824] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:23.996248 27479 tablet_bootstrap.cc:492] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b: No bootstrap required, opened a new log
I20260619 20:36:23.996248 27478 tablet_bootstrap.cc:492] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987: No bootstrap required, opened a new log
I20260619 20:36:23.996311 27479 ts_tablet_manager.cc:1397] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:36:23.996311 27478 ts_tablet_manager.cc:1397] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:36:23.996456 27480 tablet_bootstrap.cc:492] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4: No bootstrap required, opened a new log
I20260619 20:36:23.996502 27480 ts_tablet_manager.cc:1397] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:36:23.998139 27480 raft_consensus.cc:348] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } }
I20260619 20:36:23.998304 27480 raft_consensus.cc:374] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:23.998353 27480 raft_consensus.cc:729] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9952969eb524d9bb6ecfd0758cedba4, State: Initialized, Role: FOLLOWER
I20260619 20:36:23.998328 27479 raft_consensus.cc:348] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } }
I20260619 20:36:23.998313 27478 raft_consensus.cc:348] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } }
I20260619 20:36:23.998466 27478 raft_consensus.cc:374] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:23.998461 27479 raft_consensus.cc:374] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:23.998497 27479 raft_consensus.cc:729] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e5e0e4edf7e4dc2bd9d9b04050c357b, State: Initialized, Role: FOLLOWER
I20260619 20:36:23.998497 27478 raft_consensus.cc:729] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9462e18558364f33830ed4ae90a9c987, State: Initialized, Role: FOLLOWER
I20260619 20:36:23.998504 27480 consensus_queue.cc:260] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } }
I20260619 20:36:23.998615 27479 consensus_queue.cc:260] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } }
I20260619 20:36:23.998628 27478 consensus_queue.cc:260] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } }
I20260619 20:36:23.998827 27480 ts_tablet_manager.cc:1428] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 20:36:23.998875 27479 ts_tablet_manager.cc:1428] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:36:23.998875 27478 ts_tablet_manager.cc:1428] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:36:23.998961 27166 heartbeater.cc:503] Master 127.25.62.190:36709 was elected leader, sending a full tablet report...
I20260619 20:36:23.998978 27457 heartbeater.cc:503] Master 127.25.62.190:36709 was elected leader, sending a full tablet report...
I20260619 20:36:23.998979 27310 heartbeater.cc:503] Master 127.25.62.190:36709 was elected leader, sending a full tablet report...
W20260619 20:36:24.035452 27311 tablet.cc:2369] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:36:24.140923 27167 tablet.cc:2369] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:36:24.194552 27458 tablet.cc:2369] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:36:24.207393 27486 raft_consensus.cc:482] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:36:24.207561 27486 raft_consensus.cc:504] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } }
I20260619 20:36:24.207948 27486 leader_election.cc:288] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9952969eb524d9bb6ecfd0758cedba4 (127.25.62.129:36935), 9462e18558364f33830ed4ae90a9c987 (127.25.62.131:43469)
I20260619 20:36:24.211793 27487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.208175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:36935 (local address 127.25.62.130:60061)
0619 20:36:24.208450 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:24.208460 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:24.208526 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:24.208673 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:24.208674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:24.208675 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:24.208741 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.208742 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.209591 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.209592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.210915 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.210919 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.211528 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.211529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.211627 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.211631 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:24.211675 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 20:36:24.211708 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":105,"thread_start_us":104,"threads_started":1}
I20260619 20:36:24.211850 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.208240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:60061 (local address 127.25.62.129:36935)
0619 20:36:24.208288 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:24.208290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:24.208543 (+ 253us) server_negotiation.cc:407] Connection header received
0619 20:36:24.208572 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:24.208573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:24.208580 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:24.208656 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:24.208807 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.209517 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.211005 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.211435 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.211503 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.211510 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:24.211716 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 20:36:24.211764 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 20:36:24.212374 27477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.208266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:45341 (local address 127.25.62.131:43469)
0619 20:36:24.208322 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:24.208324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:24.208335 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:36:24.208369 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:24.208370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:24.208380 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:24.208458 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:24.208642 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.209449 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.211024 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.211451 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.211552 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.211562 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:24.212288 (+ 726us) server_negotiation.cc:299] Negotiation successful
0619 20:36:24.212327 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 20:36:24.212299 27120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e0e2a871ea476683c2fb7e05e8cc6c" candidate_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9952969eb524d9bb6ecfd0758cedba4" is_pre_election: true
I20260619 20:36:24.212383 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.208175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:43469 (local address 127.25.62.130:45341)
0619 20:36:24.208243 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:24.208256 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:24.208307 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:24.208479 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:24.208479 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:24.208482 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:24.208563 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.208566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.209546 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.209547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.210931 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.210933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.211594 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.211597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.212201 (+ 604us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.212206 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:36:24.212274 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 20:36:24.212317 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":41}
I20260619 20:36:24.212513 27120 raft_consensus.cc:2454] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e5e0e4edf7e4dc2bd9d9b04050c357b in term 0.
I20260619 20:36:24.212816 27192 leader_election.cc:302] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e5e0e4edf7e4dc2bd9d9b04050c357b, b9952969eb524d9bb6ecfd0758cedba4; no voters:
I20260619 20:36:24.212831 27409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e0e2a871ea476683c2fb7e05e8cc6c" candidate_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9462e18558364f33830ed4ae90a9c987" is_pre_election: true
I20260619 20:36:24.212975 27486 raft_consensus.cc:2781] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:36:24.212997 27409 raft_consensus.cc:2454] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e5e0e4edf7e4dc2bd9d9b04050c357b in term 0.
I20260619 20:36:24.213058 27486 raft_consensus.cc:482] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:36:24.213111 27486 raft_consensus.cc:3037] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:24.213851 27486 raft_consensus.cc:504] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } }
I20260619 20:36:24.214010 27486 leader_election.cc:288] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [CANDIDATE]: Term 1 election: Requested vote from peers b9952969eb524d9bb6ecfd0758cedba4 (127.25.62.129:36935), 9462e18558364f33830ed4ae90a9c987 (127.25.62.131:43469)
I20260619 20:36:24.214187 27120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e0e2a871ea476683c2fb7e05e8cc6c" candidate_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9952969eb524d9bb6ecfd0758cedba4"
I20260619 20:36:24.214282 27120 raft_consensus.cc:3037] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:24.214342 27409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e0e2a871ea476683c2fb7e05e8cc6c" candidate_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9462e18558364f33830ed4ae90a9c987"
I20260619 20:36:24.214430 27409 raft_consensus.cc:3037] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:24.215054 27120 raft_consensus.cc:2454] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e5e0e4edf7e4dc2bd9d9b04050c357b in term 1.
I20260619 20:36:24.215229 27192 leader_election.cc:302] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e5e0e4edf7e4dc2bd9d9b04050c357b, b9952969eb524d9bb6ecfd0758cedba4; no voters:
I20260619 20:36:24.215314 27409 raft_consensus.cc:2454] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e5e0e4edf7e4dc2bd9d9b04050c357b in term 1.
I20260619 20:36:24.215363 27486 raft_consensus.cc:2781] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:24.215534 27486 raft_consensus.cc:686] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 1 LEADER]: Becoming Leader. State: Replica: 6e5e0e4edf7e4dc2bd9d9b04050c357b, State: Running, Role: LEADER
I20260619 20:36:24.215639 27486 consensus_queue.cc:237] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } }
I20260619 20:36:24.216497 26963 catalog_manager.cc:5495] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e5e0e4edf7e4dc2bd9d9b04050c357b (127.25.62.130). New cstate: current_term: 1 leader_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } health_report { overall_health: UNKNOWN } } }
I20260619 20:36:24.272954 27476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.270206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45212 (local address 127.25.62.130:35913)
0619 20:36:24.270272 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:24.270274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:24.270283 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:36:24.270359 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:24.270360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:24.270380 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:24.270461 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:24.270641 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.271191 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.272071 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.272281 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.272365 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.272586 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:24.272693 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:24.272788 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:24.272829 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 20:36:24.272850 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 20:36:24.274745 27409 raft_consensus.cc:1261] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987 [term 1 FOLLOWER]: Refusing update from remote peer 6e5e0e4edf7e4dc2bd9d9b04050c357b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:24.274793 27120 raft_consensus.cc:1261] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4 [term 1 FOLLOWER]: Refusing update from remote peer 6e5e0e4edf7e4dc2bd9d9b04050c357b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:24.275187 27486 consensus_queue.cc:1035] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:24.275349 27488 consensus_queue.cc:1035] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:24.277860 27491 mvcc.cc:205] Tried to move back new op lower bound from 7298668069986545664 to 7298668069747875840. Current Snapshot: MvccSnapshot[applied={T|T < 7298668069986545664}]
I20260619 20:36:24.277882 27492 mvcc.cc:205] Tried to move back new op lower bound from 7298668069986545664 to 7298668069747875840. Current Snapshot: MvccSnapshot[applied={T|T < 7298668069986545664}]
I20260619 20:36:24.278934 27493 mvcc.cc:205] Tried to move back new op lower bound from 7298668069986545664 to 7298668069747875840. Current Snapshot: MvccSnapshot[applied={T|T < 7298668069986545664 or (T in {7298668069986545664})}]
I20260619 20:36:24.281730 26961 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:24.281823 26961 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:24.282637 26961 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456: Sending DeleteTablet for 3 replicas of tablet d3e0e2a871ea476683c2fb7e05e8cc6c
I20260619 20:36:24.282985 27100 tablet_service.cc:1514] Processing DeleteTablet for tablet d3e0e2a871ea476683c2fb7e05e8cc6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:24 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:54636
I20260619 20:36:24.282985 27389 tablet_service.cc:1514] Processing DeleteTablet for tablet d3e0e2a871ea476683c2fb7e05e8cc6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:24 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:42460
I20260619 20:36:24.283193 26961 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 20:36:24.283255 27240 tablet_service.cc:1514] Processing DeleteTablet for tablet d3e0e2a871ea476683c2fb7e05e8cc6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:24 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:45198
I20260619 20:36:24.283285 27502 tablet_replica.cc:330] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4: stopping tablet replica
W20260619 20:36:24.283330 26961 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:36:24.283375 27502 raft_consensus.cc:2229] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:24.283463 27502 raft_consensus.cc:2258] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:24.283623 27503 tablet_replica.cc:330] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987: stopping tablet replica
I20260619 20:36:24.283712 27503 raft_consensus.cc:2229] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:24.283807 27503 raft_consensus.cc:2258] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:24.283924 27502 ts_tablet_manager.cc:1905] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:24.284219 27503 ts_tablet_manager.cc:1905] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:24.285017 27240 tablet_service.cc:1467] Processing CreateTablet for tablet 4e463a622075450490c3705fc0bb7221 (DEFAULT_TABLE table=test-table [id=ded6548e04d244a08c78a199033e7134]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:24.285151 27240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e463a622075450490c3705fc0bb7221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:24.285887 27502 ts_tablet_manager.cc:1918] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:36:24.285949 27502 log.cc:1196] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-0/wal/wals/d3e0e2a871ea476683c2fb7e05e8cc6c
I20260619 20:36:24.286221 27502 ts_tablet_manager.cc:1939] T d3e0e2a871ea476683c2fb7e05e8cc6c P b9952969eb524d9bb6ecfd0758cedba4: Deleting consensus metadata
I20260619 20:36:24.286329 27479 tablet_bootstrap.cc:492] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Bootstrap starting.
I20260619 20:36:24.286388 27503 ts_tablet_manager.cc:1918] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:36:24.286428 27389 tablet_service.cc:1467] Processing CreateTablet for tablet 4e463a622075450490c3705fc0bb7221 (DEFAULT_TABLE table=test-table [id=ded6548e04d244a08c78a199033e7134]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:24.286543 27389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e463a622075450490c3705fc0bb7221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:24.286777 27479 tablet_bootstrap.cc:654] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:24.287402 27479 tablet_bootstrap.cc:492] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b: No bootstrap required, opened a new log
I20260619 20:36:24.287446 27479 ts_tablet_manager.cc:1397] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:24.287521 27478 tablet_bootstrap.cc:492] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987: Bootstrap starting.
I20260619 20:36:24.287573 27479 raft_consensus.cc:348] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } }
I20260619 20:36:24.287643 27479 raft_consensus.cc:374] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:24.287664 27479 raft_consensus.cc:729] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e5e0e4edf7e4dc2bd9d9b04050c357b, State: Initialized, Role: FOLLOWER
I20260619 20:36:24.287703 27479 consensus_queue.cc:260] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } }
I20260619 20:36:24.287779 27479 ts_tablet_manager.cc:1428] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:24.287843 26948 catalog_manager.cc:4841] TS b9952969eb524d9bb6ecfd0758cedba4 (127.25.62.129:36935): tablet d3e0e2a871ea476683c2fb7e05e8cc6c (table test-table [id=98edb63443a349598ab5b3e435a5648b]) successfully deleted
I20260619 20:36:24.287941 27478 tablet_bootstrap.cc:654] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:24.288269 27100 tablet_service.cc:1467] Processing CreateTablet for tablet 4e463a622075450490c3705fc0bb7221 (DEFAULT_TABLE table=test-table [id=ded6548e04d244a08c78a199033e7134]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:24.288404 27100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e463a622075450490c3705fc0bb7221. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:24.286440 27503 log.cc:1196] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/d3e0e2a871ea476683c2fb7e05e8cc6c
I20260619 20:36:24.289511 27480 tablet_bootstrap.cc:492] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4: Bootstrap starting.
I20260619 20:36:24.289535 27503 ts_tablet_manager.cc:1939] T d3e0e2a871ea476683c2fb7e05e8cc6c P 9462e18558364f33830ed4ae90a9c987: Deleting consensus metadata
I20260619 20:36:24.289887 27480 tablet_bootstrap.cc:654] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:24.289934 26947 catalog_manager.cc:4841] TS 9462e18558364f33830ed4ae90a9c987 (127.25.62.131:43469): tablet d3e0e2a871ea476683c2fb7e05e8cc6c (table test-table [id=98edb63443a349598ab5b3e435a5648b]) successfully deleted
I20260619 20:36:24.290058 27504 tablet_replica.cc:330] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b: stopping tablet replica
I20260619 20:36:24.290144 27504 raft_consensus.cc:2229] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:36:24.290153 27478 tablet_bootstrap.cc:492] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987: No bootstrap required, opened a new log
I20260619 20:36:24.290195 27478 ts_tablet_manager.cc:1397] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 20:36:24.290230 27504 raft_consensus.cc:2258] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:24.290346 27478 raft_consensus.cc:348] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } }
I20260619 20:36:24.290424 27478 raft_consensus.cc:374] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:24.290450 27478 raft_consensus.cc:729] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9462e18558364f33830ed4ae90a9c987, State: Initialized, Role: FOLLOWER
I20260619 20:36:24.290506 27478 consensus_queue.cc:260] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } }
I20260619 20:36:24.290591 27478 ts_tablet_manager.cc:1428] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:24.290669 27480 tablet_bootstrap.cc:492] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4: No bootstrap required, opened a new log
I20260619 20:36:24.290709 27480 ts_tablet_manager.cc:1397] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:24.291028 27480 raft_consensus.cc:348] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } }
I20260619 20:36:24.291113 27480 raft_consensus.cc:374] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:24.291136 27480 raft_consensus.cc:729] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9952969eb524d9bb6ecfd0758cedba4, State: Initialized, Role: FOLLOWER
I20260619 20:36:24.291195 27480 consensus_queue.cc:260] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } }
I20260619 20:36:24.291287 27480 ts_tablet_manager.cc:1428] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:24.291627 27504 ts_tablet_manager.cc:1905] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:24.293125 27504 ts_tablet_manager.cc:1918] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:36:24.293192 27504 log.cc:1196] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/d3e0e2a871ea476683c2fb7e05e8cc6c
I20260619 20:36:24.293438 27504 ts_tablet_manager.cc:1939] T d3e0e2a871ea476683c2fb7e05e8cc6c P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Deleting consensus metadata
I20260619 20:36:24.293855 26947 catalog_manager.cc:4841] TS 6e5e0e4edf7e4dc2bd9d9b04050c357b (127.25.62.130:35913): tablet d3e0e2a871ea476683c2fb7e05e8cc6c (table test-table [id=98edb63443a349598ab5b3e435a5648b]) successfully deleted
I20260619 20:36:24.596279 27485 raft_consensus.cc:482] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:36:24.596396 27485 raft_consensus.cc:504] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } }
I20260619 20:36:24.596738 27485 leader_election.cc:288] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9952969eb524d9bb6ecfd0758cedba4 (127.25.62.129:36935), 6e5e0e4edf7e4dc2bd9d9b04050c357b (127.25.62.130:35913)
I20260619 20:36:24.601109 27476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.597185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:35421 (local address 127.25.62.130:35913)
0619 20:36:24.597234 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:24.597236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:24.597390 (+ 154us) server_negotiation.cc:407] Connection header received
0619 20:36:24.597434 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:24.597436 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:24.597450 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:24.597527 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:24.597772 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.598518 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.600137 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.600682 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.600759 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.600768 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:24.600986 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 20:36:24.601025 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:36:24.601109 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.597185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:58583 (local address 127.25.62.129:36935)
0619 20:36:24.597234 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:24.597236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:24.597390 (+ 154us) server_negotiation.cc:407] Connection header received
0619 20:36:24.597434 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:24.597436 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:24.597448 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:24.597527 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:24.597772 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.598518 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.600137 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.600682 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.600759 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.600768 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:24.600984 (+ 216us) server_negotiation.cc:299] Negotiation successful
0619 20:36:24.601020 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:36:24.601135 27509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.597119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:36935 (local address 127.25.62.131:58583)
0619 20:36:24.597265 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:24.597278 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:24.597358 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:24.597562 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:24.597563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:24.597567 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:24.597668 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.597671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.598624 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.598625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.600034 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.600036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.600789 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.600790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.600877 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.600882 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:36:24.600964 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 20:36:24.601003 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":131,"thread_start_us":76,"threads_started":1}
I20260619 20:36:24.601145 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.597128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:35913 (local address 127.25.62.131:35421)
0619 20:36:24.597285 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:24.597293 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:24.597358 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:24.597562 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:24.597563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:24.597566 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:24.597668 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.597670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.598623 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.598624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.600029 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.600033 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.600790 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.600791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.600877 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.600883 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 20:36:24.600952 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 20:36:24.601003 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":97,"spinlock_wait_cycles":1280,"thread_start_us":73,"threads_started":1}
I20260619 20:36:24.601534 27120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e463a622075450490c3705fc0bb7221" candidate_uuid: "9462e18558364f33830ed4ae90a9c987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9952969eb524d9bb6ecfd0758cedba4" is_pre_election: true
I20260619 20:36:24.601641 27120 raft_consensus.cc:2454] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9462e18558364f33830ed4ae90a9c987 in term 0.
I20260619 20:36:24.601688 27262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e463a622075450490c3705fc0bb7221" candidate_uuid: "9462e18558364f33830ed4ae90a9c987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" is_pre_election: true
I20260619 20:36:24.601822 27262 raft_consensus.cc:2454] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9462e18558364f33830ed4ae90a9c987 in term 0.
I20260619 20:36:24.602057 27338 leader_election.cc:302] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9462e18558364f33830ed4ae90a9c987, b9952969eb524d9bb6ecfd0758cedba4; no voters:
I20260619 20:36:24.602269 27485 raft_consensus.cc:2781] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:36:24.602320 27485 raft_consensus.cc:482] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:36:24.602340 27485 raft_consensus.cc:3037] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:24.603040 27485 raft_consensus.cc:504] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } }
I20260619 20:36:24.603195 27485 leader_election.cc:288] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [CANDIDATE]: Term 1 election: Requested vote from peers b9952969eb524d9bb6ecfd0758cedba4 (127.25.62.129:36935), 6e5e0e4edf7e4dc2bd9d9b04050c357b (127.25.62.130:35913)
I20260619 20:36:24.603341 27120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e463a622075450490c3705fc0bb7221" candidate_uuid: "9462e18558364f33830ed4ae90a9c987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9952969eb524d9bb6ecfd0758cedba4"
I20260619 20:36:24.603348 27262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e463a622075450490c3705fc0bb7221" candidate_uuid: "9462e18558364f33830ed4ae90a9c987" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b"
I20260619 20:36:24.603442 27120 raft_consensus.cc:3037] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:24.603441 27262 raft_consensus.cc:3037] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:24.604048 27120 raft_consensus.cc:2454] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9462e18558364f33830ed4ae90a9c987 in term 1.
I20260619 20:36:24.604250 27262 raft_consensus.cc:2454] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9462e18558364f33830ed4ae90a9c987 in term 1.
I20260619 20:36:24.604265 27338 leader_election.cc:302] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9462e18558364f33830ed4ae90a9c987, b9952969eb524d9bb6ecfd0758cedba4; no voters:
I20260619 20:36:24.604391 27485 raft_consensus.cc:2781] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:24.604497 27485 raft_consensus.cc:686] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [term 1 LEADER]: Becoming Leader. State: Replica: 9462e18558364f33830ed4ae90a9c987, State: Running, Role: LEADER
I20260619 20:36:24.604565 27485 consensus_queue.cc:237] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } }
I20260619 20:36:24.605299 26963 catalog_manager.cc:5495] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9462e18558364f33830ed4ae90a9c987 (127.25.62.131). New cstate: current_term: 1 leader_uuid: "9462e18558364f33830ed4ae90a9c987" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } health_report { overall_health: HEALTHY } } }
Jun 19 20:36:24 dist-test-slave-0t40 krb5kdc[26918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901383, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:24.674312 27518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.666394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45298 (local address 127.25.62.190:36709)
0619 20:36:24.666542 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:24.666544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:24.666552 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:24.666577 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:24.666578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:24.666589 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:24.666659 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:24.667346 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.668207 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.669224 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.669520 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.669581 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.671710 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:24.671711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:24.671711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:24.671718 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:24.673092 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:24.673463 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:24.673464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:24.673464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:24.673524 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:24.673736 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:24.673737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:24.673737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:24.673851 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:24.673939 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:24.674076 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 20:36:24.674178 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":119,"thread_start_us":96,"threads_started":1}
I20260619 20:36:24.674360 27517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.665836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:36709 (local address 127.0.0.1:45298)
0619 20:36:24.666015 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:24.666026 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:24.666086 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:24.666683 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:24.666685 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:24.667163 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:24.667225 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.667228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.668327 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.668328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.669130 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.669132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.669634 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.669635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.669684 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.670144 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:24.670148 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:24.671618 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:24.673152 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:24.673160 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:24.673163 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:24.673364 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:24.673599 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:24.673600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:24.673601 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:24.673658 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:24.673922 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:24.673930 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:24.673974 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 20:36:24.674047 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 20:36:24.674227 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":79,"thread_start_us":84,"threads_started":1}
I20260619 20:36:24.676327 26963 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:45298:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 20:36:24.676528 26963 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:36:24.678046 27389 tablet_service.cc:1467] Processing CreateTablet for tablet 44e44cdd6f4e4b24b2be1386af66e96a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=742437ebf09543a68fd48d4fb1315031]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:36:24.678053 27100 tablet_service.cc:1467] Processing CreateTablet for tablet 44e44cdd6f4e4b24b2be1386af66e96a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=742437ebf09543a68fd48d4fb1315031]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:36:24.678210 27389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e44cdd6f4e4b24b2be1386af66e96a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:24.678210 27100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e44cdd6f4e4b24b2be1386af66e96a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:24.679426 27480 tablet_bootstrap.cc:492] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4: Bootstrap starting.
I20260619 20:36:24.679484 27478 tablet_bootstrap.cc:492] T 44e44cdd6f4e4b24b2be1386af66e96a P 9462e18558364f33830ed4ae90a9c987: Bootstrap starting.
I20260619 20:36:24.679600 27240 tablet_service.cc:1467] Processing CreateTablet for tablet 44e44cdd6f4e4b24b2be1386af66e96a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=742437ebf09543a68fd48d4fb1315031]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:36:24.679723 27240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e44cdd6f4e4b24b2be1386af66e96a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:24.679749 27480 tablet_bootstrap.cc:654] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:24.679827 27478 tablet_bootstrap.cc:654] T 44e44cdd6f4e4b24b2be1386af66e96a P 9462e18558364f33830ed4ae90a9c987: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:24.680374 27480 tablet_bootstrap.cc:492] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4: No bootstrap required, opened a new log
I20260619 20:36:24.680431 27480 ts_tablet_manager.cc:1397] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:24.680591 27480 raft_consensus.cc:348] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } }
I20260619 20:36:24.680677 27480 raft_consensus.cc:374] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:24.680698 27480 raft_consensus.cc:729] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9952969eb524d9bb6ecfd0758cedba4, State: Initialized, Role: FOLLOWER
I20260619 20:36:24.680740 27480 consensus_queue.cc:260] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } }
I20260619 20:36:24.680828 27480 ts_tablet_manager.cc:1428] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:24.680871 27484 tablet_replica.cc:439] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } }
I20260619 20:36:24.680968 27484 tablet_replica.cc:442] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:24.681143 27478 tablet_bootstrap.cc:492] T 44e44cdd6f4e4b24b2be1386af66e96a P 9462e18558364f33830ed4ae90a9c987: No bootstrap required, opened a new log
I20260619 20:36:24.681181 27478 ts_tablet_manager.cc:1397] T 44e44cdd6f4e4b24b2be1386af66e96a P 9462e18558364f33830ed4ae90a9c987: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:36:24.681263 27479 tablet_bootstrap.cc:492] T 44e44cdd6f4e4b24b2be1386af66e96a P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Bootstrap starting.
I20260619 20:36:24.681319 27478 raft_consensus.cc:348] T 44e44cdd6f4e4b24b2be1386af66e96a P 9462e18558364f33830ed4ae90a9c987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } }
I20260619 20:36:24.681391 27478 raft_consensus.cc:374] T 44e44cdd6f4e4b24b2be1386af66e96a P 9462e18558364f33830ed4ae90a9c987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:24.681413 27478 raft_consensus.cc:729] T 44e44cdd6f4e4b24b2be1386af66e96a P 9462e18558364f33830ed4ae90a9c987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9462e18558364f33830ed4ae90a9c987, State: Initialized, Role: FOLLOWER
I20260619 20:36:24.681453 27478 consensus_queue.cc:260] T 44e44cdd6f4e4b24b2be1386af66e96a P 9462e18558364f33830ed4ae90a9c987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } }
I20260619 20:36:24.681535 27478 ts_tablet_manager.cc:1428] T 44e44cdd6f4e4b24b2be1386af66e96a P 9462e18558364f33830ed4ae90a9c987: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:24.681596 27479 tablet_bootstrap.cc:654] T 44e44cdd6f4e4b24b2be1386af66e96a P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:24.681602 27510 tablet_replica.cc:439] T 44e44cdd6f4e4b24b2be1386af66e96a P 9462e18558364f33830ed4ae90a9c987: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } }
I20260619 20:36:24.681699 27510 tablet_replica.cc:442] T 44e44cdd6f4e4b24b2be1386af66e96a P 9462e18558364f33830ed4ae90a9c987: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:24.682207 27479 tablet_bootstrap.cc:492] T 44e44cdd6f4e4b24b2be1386af66e96a P 6e5e0e4edf7e4dc2bd9d9b04050c357b: No bootstrap required, opened a new log
I20260619 20:36:24.682256 27479 ts_tablet_manager.cc:1397] T 44e44cdd6f4e4b24b2be1386af66e96a P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:24.682365 27479 raft_consensus.cc:348] T 44e44cdd6f4e4b24b2be1386af66e96a P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } }
I20260619 20:36:24.682437 27479 raft_consensus.cc:374] T 44e44cdd6f4e4b24b2be1386af66e96a P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:24.682473 27479 raft_consensus.cc:729] T 44e44cdd6f4e4b24b2be1386af66e96a P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e5e0e4edf7e4dc2bd9d9b04050c357b, State: Initialized, Role: FOLLOWER
I20260619 20:36:24.682523 27479 consensus_queue.cc:260] T 44e44cdd6f4e4b24b2be1386af66e96a P 6e5e0e4edf7e4dc2bd9d9b04050c357b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } }
I20260619 20:36:24.682614 27479 ts_tablet_manager.cc:1428] T 44e44cdd6f4e4b24b2be1386af66e96a P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:24.682689 27498 tablet_replica.cc:439] T 44e44cdd6f4e4b24b2be1386af66e96a P 6e5e0e4edf7e4dc2bd9d9b04050c357b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } }
I20260619 20:36:24.682786 27498 tablet_replica.cc:442] T 44e44cdd6f4e4b24b2be1386af66e96a P 6e5e0e4edf7e4dc2bd9d9b04050c357b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:24.702556 27484 raft_consensus.cc:482] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:36:24.702652 27484 raft_consensus.cc:504] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } }
I20260619 20:36:24.702941 27484 leader_election.cc:288] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9462e18558364f33830ed4ae90a9c987 (127.25.62.131:43469), 6e5e0e4edf7e4dc2bd9d9b04050c357b (127.25.62.130:35913)
I20260619 20:36:24.706157 27477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.703235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:44931 (local address 127.25.62.131:43469)
0619 20:36:24.703324 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:24.703326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:24.703408 (+ 82us) server_negotiation.cc:407] Connection header received
0619 20:36:24.703441 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:24.703442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:24.703451 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:24.703539 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:24.703773 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.704318 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.705408 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.705775 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.705832 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.705839 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:24.706041 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 20:36:24.706090 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 20:36:24.706151 27522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.703154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:43469 (local address 127.25.62.129:44931)
0619 20:36:24.703314 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:24.703325 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:24.703384 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:24.703570 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:24.703571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:24.703574 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:24.703683 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.703685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.704395 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.704396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.705321 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.705323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.705857 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.705859 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.705932 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.705938 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 20:36:24.705997 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 20:36:24.706046 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":1,"thread_start_us":111,"threads_started":1}
I20260619 20:36:24.706234 27476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.703510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:42277 (local address 127.25.62.130:35913)
0619 20:36:24.703550 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:24.703552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:24.703582 (+ 30us) server_negotiation.cc:407] Connection header received
0619 20:36:24.703673 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:24.703674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:24.703683 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:24.703742 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:24.703922 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.704405 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.705442 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.705793 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.705883 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.705891 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:24.706117 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 20:36:24.706165 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 20:36:24.706331 27523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.703383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:35913 (local address 127.25.62.129:42277)
0619 20:36:24.703526 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:24.703536 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:24.703598 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:24.703773 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:24.703774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:24.703775 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:24.703854 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.703856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.704511 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.704512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.705324 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.705327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.705913 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.705915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.706001 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.706007 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 20:36:24.706103 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 20:36:24.706143 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":15,"thread_start_us":67,"threads_started":1}
I20260619 20:36:24.706601 27409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e44cdd6f4e4b24b2be1386af66e96a" candidate_uuid: "b9952969eb524d9bb6ecfd0758cedba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9462e18558364f33830ed4ae90a9c987" is_pre_election: true
I20260619 20:36:24.706584 27262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e44cdd6f4e4b24b2be1386af66e96a" candidate_uuid: "b9952969eb524d9bb6ecfd0758cedba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" is_pre_election: true
I20260619 20:36:24.706679 27409 raft_consensus.cc:2454] T 44e44cdd6f4e4b24b2be1386af66e96a P 9462e18558364f33830ed4ae90a9c987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9952969eb524d9bb6ecfd0758cedba4 in term 0.
I20260619 20:36:24.706681 27262 raft_consensus.cc:2454] T 44e44cdd6f4e4b24b2be1386af66e96a P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9952969eb524d9bb6ecfd0758cedba4 in term 0.
I20260619 20:36:24.706851 27051 leader_election.cc:302] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9462e18558364f33830ed4ae90a9c987, b9952969eb524d9bb6ecfd0758cedba4; no voters:
I20260619 20:36:24.707005 27484 raft_consensus.cc:2781] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:36:24.707055 27484 raft_consensus.cc:482] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:36:24.707087 27484 raft_consensus.cc:3037] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:24.707788 27484 raft_consensus.cc:504] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } }
I20260619 20:36:24.707930 27484 leader_election.cc:288] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 [CANDIDATE]: Term 1 election: Requested vote from peers 9462e18558364f33830ed4ae90a9c987 (127.25.62.131:43469), 6e5e0e4edf7e4dc2bd9d9b04050c357b (127.25.62.130:35913)
I20260619 20:36:24.708139 27262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e44cdd6f4e4b24b2be1386af66e96a" candidate_uuid: "b9952969eb524d9bb6ecfd0758cedba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b"
I20260619 20:36:24.708151 27409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e44cdd6f4e4b24b2be1386af66e96a" candidate_uuid: "b9952969eb524d9bb6ecfd0758cedba4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9462e18558364f33830ed4ae90a9c987"
I20260619 20:36:24.708237 27409 raft_consensus.cc:3037] T 44e44cdd6f4e4b24b2be1386af66e96a P 9462e18558364f33830ed4ae90a9c987 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:24.708240 27262 raft_consensus.cc:3037] T 44e44cdd6f4e4b24b2be1386af66e96a P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:24.708827 27262 raft_consensus.cc:2454] T 44e44cdd6f4e4b24b2be1386af66e96a P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9952969eb524d9bb6ecfd0758cedba4 in term 1.
I20260619 20:36:24.708833 27409 raft_consensus.cc:2454] T 44e44cdd6f4e4b24b2be1386af66e96a P 9462e18558364f33830ed4ae90a9c987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9952969eb524d9bb6ecfd0758cedba4 in term 1.
I20260619 20:36:24.708990 27051 leader_election.cc:302] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e5e0e4edf7e4dc2bd9d9b04050c357b, b9952969eb524d9bb6ecfd0758cedba4; no voters:
I20260619 20:36:24.709136 27484 raft_consensus.cc:2781] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:24.709295 27484 raft_consensus.cc:686] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 [term 1 LEADER]: Becoming Leader. State: Replica: b9952969eb524d9bb6ecfd0758cedba4, State: Running, Role: LEADER
I20260619 20:36:24.709369 27484 consensus_queue.cc:237] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } }
I20260619 20:36:24.709740 27524 tablet_replica.cc:439] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4: TxnStatusTablet state changed. Reason: New leader b9952969eb524d9bb6ecfd0758cedba4. Latest consensus state: current_term: 1 leader_uuid: "b9952969eb524d9bb6ecfd0758cedba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } }
I20260619 20:36:24.709816 27524 tablet_replica.cc:442] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:36:24.710052 27526 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 20:36:24.710078 26963 catalog_manager.cc:5495] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b9952969eb524d9bb6ecfd0758cedba4 (127.25.62.129). New cstate: current_term: 1 leader_uuid: "b9952969eb524d9bb6ecfd0758cedba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } health_report { overall_health: UNKNOWN } } }
I20260619 20:36:24.710170 27526 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 20:36:24.719456 27517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.716549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:36935 (local address 127.0.0.1:54640)
0619 20:36:24.716632 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:24.716642 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:24.716710 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:24.716907 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:24.716908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:24.716911 (+ 3us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:36:24.716966 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.716968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.717895 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.717896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.718468 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.718470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.718961 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.718962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.719016 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.719030 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:36:24.719321 (+ 291us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:36:24.719328 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 20:36:24.719364 (+ 36us) client_negotiation.cc:240] Negotiation successful
0619 20:36:24.719402 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 20:36:24.719475 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.716602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54640 (local address 127.25.62.129:36935)
0619 20:36:24.716641 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:24.716643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:24.716682 (+ 39us) server_negotiation.cc:407] Connection header received
0619 20:36:24.716797 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:24.716797 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:24.716816 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:24.716900 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:24.717079 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.717806 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.718555 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.718846 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.718920 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.719141 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:24.719250 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:24.719325 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:24.719368 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 20:36:24.719387 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:36:24.720826 27409 raft_consensus.cc:1261] T 44e44cdd6f4e4b24b2be1386af66e96a P 9462e18558364f33830ed4ae90a9c987 [term 1 FOLLOWER]: Refusing update from remote peer b9952969eb524d9bb6ecfd0758cedba4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:24.720891 27262 raft_consensus.cc:1261] T 44e44cdd6f4e4b24b2be1386af66e96a P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 1 FOLLOWER]: Refusing update from remote peer b9952969eb524d9bb6ecfd0758cedba4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:24.721135 27484 consensus_queue.cc:1035] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:24.721254 27524 consensus_queue.cc:1035] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:24.721758 27510 tablet_replica.cc:439] T 44e44cdd6f4e4b24b2be1386af66e96a P 9462e18558364f33830ed4ae90a9c987: TxnStatusTablet state changed. Reason: New leader b9952969eb524d9bb6ecfd0758cedba4. Latest consensus state: current_term: 1 leader_uuid: "b9952969eb524d9bb6ecfd0758cedba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } }
I20260619 20:36:24.721863 27510 tablet_replica.cc:442] T 44e44cdd6f4e4b24b2be1386af66e96a P 9462e18558364f33830ed4ae90a9c987: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:24.722357 27498 tablet_replica.cc:439] T 44e44cdd6f4e4b24b2be1386af66e96a P 6e5e0e4edf7e4dc2bd9d9b04050c357b: TxnStatusTablet state changed. Reason: New leader b9952969eb524d9bb6ecfd0758cedba4. Latest consensus state: current_term: 1 leader_uuid: "b9952969eb524d9bb6ecfd0758cedba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } }
I20260619 20:36:24.722453 27498 tablet_replica.cc:442] T 44e44cdd6f4e4b24b2be1386af66e96a P 6e5e0e4edf7e4dc2bd9d9b04050c357b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:24.722536 27524 tablet_replica.cc:439] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b9952969eb524d9bb6ecfd0758cedba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } }
I20260619 20:36:24.722610 27524 tablet_replica.cc:442] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:36:24.723119 27510 tablet_replica.cc:439] T 44e44cdd6f4e4b24b2be1386af66e96a P 9462e18558364f33830ed4ae90a9c987: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b9952969eb524d9bb6ecfd0758cedba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } }
I20260619 20:36:24.723198 27510 tablet_replica.cc:442] T 44e44cdd6f4e4b24b2be1386af66e96a P 9462e18558364f33830ed4ae90a9c987: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:24.723376 27484 tablet_replica.cc:439] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b9952969eb524d9bb6ecfd0758cedba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } }
I20260619 20:36:24.723477 27484 tablet_replica.cc:442] T 44e44cdd6f4e4b24b2be1386af66e96a P b9952969eb524d9bb6ecfd0758cedba4: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:36:24.723798 27498 tablet_replica.cc:439] T 44e44cdd6f4e4b24b2be1386af66e96a P 6e5e0e4edf7e4dc2bd9d9b04050c357b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b9952969eb524d9bb6ecfd0758cedba4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 } } peers { permanent_uuid: "9462e18558364f33830ed4ae90a9c987" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43469 } } peers { permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 } } }
I20260619 20:36:24.723869 27498 tablet_replica.cc:442] T 44e44cdd6f4e4b24b2be1386af66e96a P 6e5e0e4edf7e4dc2bd9d9b04050c357b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:24.727167 27477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.724675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42474 (local address 127.25.62.131:43469)
0619 20:36:24.724713 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:24.724716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:24.724724 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:24.724782 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:24.724783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:24.724803 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:24.724875 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:24.725012 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.725510 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.726244 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.726460 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.726538 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.726721 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:24.726828 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:24.726908 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:24.726992 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 20:36:24.727012 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:36:24.732472 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.729866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:56153 (local address 127.25.62.129:36935)
0619 20:36:24.729910 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:24.729913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:24.730031 (+ 118us) server_negotiation.cc:407] Connection header received
0619 20:36:24.730084 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:24.730085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:24.730093 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:24.730143 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:24.730267 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.730690 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.731812 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.732002 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.732063 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.732262 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:24.732289 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:24.732331 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:24.732408 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 20:36:24.732425 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:36:24.732543 27533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.729749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:36935 (local address 127.25.62.131:56153)
0619 20:36:24.729956 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:24.729965 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:24.730025 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:24.730147 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:24.730148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:24.730150 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:36:24.730209 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.730210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.730764 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.730764 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.731716 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.731719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.732101 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.732102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.732167 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.732181 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:36:24.732352 (+ 171us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:36:24.732353 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 20:36:24.732406 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:36:24.732448 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
I20260619 20:36:24.735096 27262 raft_consensus.cc:1261] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 1 FOLLOWER]: Refusing update from remote peer 9462e18558364f33830ed4ae90a9c987: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:24.735188 27120 raft_consensus.cc:1261] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4 [term 1 FOLLOWER]: Refusing update from remote peer 9462e18558364f33830ed4ae90a9c987: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:24.735323 27510 consensus_queue.cc:1035] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e5e0e4edf7e4dc2bd9d9b04050c357b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:24.735474 27485 consensus_queue.cc:1035] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9952969eb524d9bb6ecfd0758cedba4" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 36935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:24.743944 27477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.740832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:59719 (local address 127.25.62.131:43469)
0619 20:36:24.740894 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:24.740896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:24.741104 (+ 208us) server_negotiation.cc:407] Connection header received
0619 20:36:24.741263 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:24.741264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:24.741274 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:24.741327 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:24.741537 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.742136 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.743117 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.743393 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.743442 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.743691 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:24.743723 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:24.743779 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:24.743849 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 20:36:24.743866 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 20:36:24.744146 27539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.740775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:43469 (local address 127.25.62.129:59719)
0619 20:36:24.741024 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:24.741055 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:24.741147 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:24.741330 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:24.741331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:24.741333 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:36:24.741453 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.741455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.742232 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.742233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.742979 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:24.742981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.743498 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.743498 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:24.743565 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.743581 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:36:24.743793 (+ 212us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:36:24.743794 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 20:36:24.743845 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 20:36:24.743883 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":72,"threads_started":1}
I20260619 20:36:24.753347 26963 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:24.753456 26963 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:24.754046 26963 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7c352c4b87f45558b0c408b35351456: Sending DeleteTablet for 3 replicas of tablet 4e463a622075450490c3705fc0bb7221
I20260619 20:36:24.754372 27240 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e463a622075450490c3705fc0bb7221 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:24 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:45198
I20260619 20:36:24.754384 27389 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e463a622075450490c3705fc0bb7221 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:24 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:42460
I20260619 20:36:24.754470 27504 tablet_replica.cc:330] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b: stopping tablet replica
I20260619 20:36:24.754551 27504 raft_consensus.cc:2229] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:24.754595 27504 raft_consensus.cc:2258] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:24.754698 27503 tablet_replica.cc:330] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987: stopping tablet replica
I20260619 20:36:24.754753 27503 raft_consensus.cc:2229] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:36:24.754838 27503 raft_consensus.cc:2258] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:24.755158 27504 ts_tablet_manager.cc:1905] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:24.755236 27503 ts_tablet_manager.cc:1905] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:24.755227 27100 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e463a622075450490c3705fc0bb7221 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:24 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:54636
I20260619 20:36:24.755323 27502 tablet_replica.cc:330] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4: stopping tablet replica
I20260619 20:36:24.755385 27502 raft_consensus.cc:2229] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:24.755419 27502 raft_consensus.cc:2258] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:24.755718 27502 ts_tablet_manager.cc:1905] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:24.757154 27504 ts_tablet_manager.cc:1918] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:36:24.757169 27503 ts_tablet_manager.cc:1918] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:36:24.757222 27503 log.cc:1196] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/4e463a622075450490c3705fc0bb7221
I20260619 20:36:24.757220 27504 log.cc:1196] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/4e463a622075450490c3705fc0bb7221
I20260619 20:36:24.757218 27502 ts_tablet_manager.cc:1918] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:36:24.757264 27502 log.cc:1196] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781901378226483-25850-0/minicluster-data/ts-0/wal/wals/4e463a622075450490c3705fc0bb7221
I20260619 20:36:24.757459 27503 ts_tablet_manager.cc:1939] T 4e463a622075450490c3705fc0bb7221 P 9462e18558364f33830ed4ae90a9c987: Deleting consensus metadata
I20260619 20:36:24.757452 27504 ts_tablet_manager.cc:1939] T 4e463a622075450490c3705fc0bb7221 P 6e5e0e4edf7e4dc2bd9d9b04050c357b: Deleting consensus metadata
I20260619 20:36:24.757472 27502 ts_tablet_manager.cc:1939] T 4e463a622075450490c3705fc0bb7221 P b9952969eb524d9bb6ecfd0758cedba4: Deleting consensus metadata
I20260619 20:36:24.757814 26947 catalog_manager.cc:4841] TS 9462e18558364f33830ed4ae90a9c987 (127.25.62.131:43469): tablet 4e463a622075450490c3705fc0bb7221 (table test-table [id=ded6548e04d244a08c78a199033e7134]) successfully deleted
I20260619 20:36:24.758060 26947 catalog_manager.cc:4841] TS 6e5e0e4edf7e4dc2bd9d9b04050c357b (127.25.62.130:35913): tablet 4e463a622075450490c3705fc0bb7221 (table test-table [id=ded6548e04d244a08c78a199033e7134]) successfully deleted
I20260619 20:36:24.758132 26948 catalog_manager.cc:4841] TS b9952969eb524d9bb6ecfd0758cedba4 (127.25.62.129:36935): tablet 4e463a622075450490c3705fc0bb7221 (table test-table [id=ded6548e04d244a08c78a199033e7134]) successfully deleted
Jun 19 20:36:24 dist-test-slave-0t40 krb5kdc[26918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901383, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.62.129@KRBTEST.COM
I20260619 20:36:24.772189 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.761335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54656 (local address 127.25.62.129:36935)
0619 20:36:24.761376 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:24.761378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:24.761592 (+ 214us) server_negotiation.cc:407] Connection header received
0619 20:36:24.761629 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:24.761630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:24.761641 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:24.761711 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:24.762536 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.763290 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.764391 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.764715 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.764795 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.768633 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:24.768645 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:24.768645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:24.768653 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:24.770742 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:24.771169 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:24.771172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:24.771173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:24.771236 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:24.771459 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:24.771459 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:24.771460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:24.771648 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:24.771813 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:24.771916 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 20:36:24.771996 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 20:36:24.772537 27160 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54656
I20260619 20:36:24.781219 27518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:24.773537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45306 (local address 127.25.62.190:36709)
0619 20:36:24.773581 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:24.773583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:24.773798 (+ 215us) server_negotiation.cc:407] Connection header received
0619 20:36:24.773831 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:24.773832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:24.773843 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:24.773903 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:24.774532 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.775273 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.776327 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:24.776660 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:24.776734 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:24.778143 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:24.778144 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:24.778145 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:24.778153 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:24.779936 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:24.780330 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:24.780331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:24.780331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:24.780390 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:24.780606 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:24.780607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:24.780607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:24.780742 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:24.780845 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:24.780981 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 20:36:24.781119 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 20:36:24.781409 26963 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45306
I20260619 20:36:24.781999 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 27035
I20260619 20:36:24.789085 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 27175
I20260619 20:36:24.795600 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 27320
I20260619 20:36:24.800235 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 26931
2026-06-19T20:36:24Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2628 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:24 dist-test-slave-0t40 krb5kdc[27548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:24 dist-test-slave-0t40 krb5kdc[27548](info): set up 2 sockets
Jun 19 20:36:24 dist-test-slave-0t40 krb5kdc[27548](info): commencing operation
krb5kdc: starting...
W20260619 20:36:25.837006 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:36:25 dist-test-slave-0t40 krb5kdc[27548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:36:25.956751 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:35573
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:39141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:35573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:26.034577 27561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:26.034798 27561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:26.034825 27561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:26.035894 27561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:26.035926 27561 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 20:36:26.035939 27561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:26.035952 27561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:26.035971 27561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:26.035983 27561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:26.037371 27561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:39141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:35573
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:35573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:26.037627 27561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:26.037887 27561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:26.042903 27566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:26.043040 27561 server_base.cc:1034] running on GCE node
W20260619 20:36:26.042906 27569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:26.042886 27567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:26.043411 27561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:26.043715 27561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:26.044898 27561 hybrid_clock.cc:648] HybridClock initialized: now 1781901386044877 us; error 56 us; skew 500 ppm
Jun 19 20:36:26 dist-test-slave-0t40 krb5kdc[27548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901386, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:26.047315 27561 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:36:26.047793 27561 webserver.cc:460] Webserver started at http://127.25.62.190:38313/ using document root <none> and password file <none>
I20260619 20:36:26.048033 27561 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:26.048106 27561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:26.048264 27561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:26.049386 27561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "71d4233cac234f918e9faa7384a1fbb9"
format_stamp: "Formatted at 2026-06-19 20:36:26 on dist-test-slave-0t40"
server_key: "93e3fc1f7686a4fa50e488f5168c7a9c"
server_key_iv: "aa747b5a348be02f116b953a33a464b2"
server_key_version: "encryptionkey@0"
I20260619 20:36:26.049743 27561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "71d4233cac234f918e9faa7384a1fbb9"
format_stamp: "Formatted at 2026-06-19 20:36:26 on dist-test-slave-0t40"
server_key: "93e3fc1f7686a4fa50e488f5168c7a9c"
server_key_iv: "aa747b5a348be02f116b953a33a464b2"
server_key_version: "encryptionkey@0"
I20260619 20:36:26.051115 27561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:26.051877 27576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:26.052104 27561 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:26.052207 27561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "71d4233cac234f918e9faa7384a1fbb9"
format_stamp: "Formatted at 2026-06-19 20:36:26 on dist-test-slave-0t40"
server_key: "93e3fc1f7686a4fa50e488f5168c7a9c"
server_key_iv: "aa747b5a348be02f116b953a33a464b2"
server_key_version: "encryptionkey@0"
I20260619 20:36:26.052294 27561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:26.067701 27561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:26.069916 27561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:26.070096 27561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:26.075146 27561 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:35573
I20260619 20:36:26.075199 27638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:35573 every 8 connection(s)
I20260619 20:36:26.075497 27561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:26.076057 27639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:26.078295 27639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9: Bootstrap starting.
I20260619 20:36:26.078881 27639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:26.079180 27639 log.cc:824] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:26.079900 27639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9: No bootstrap required, opened a new log
I20260619 20:36:26.081423 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 27561
I20260619 20:36:26.081498 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:36:26.081702 25850 external_mini_cluster.cc:1428] Setting key b9c9d6355cac8ed07acea2df3ca650b6
I20260619 20:36:26.082437 27639 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71d4233cac234f918e9faa7384a1fbb9" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35573 } }
I20260619 20:36:26.082665 27639 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:26.082702 27639 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71d4233cac234f918e9faa7384a1fbb9, State: Initialized, Role: FOLLOWER
I20260619 20:36:26.082836 27639 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71d4233cac234f918e9faa7384a1fbb9" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35573 } }
I20260619 20:36:26.082926 27639 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:26.083015 27639 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:26.083069 27639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:26.083630 27639 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71d4233cac234f918e9faa7384a1fbb9" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35573 } }
I20260619 20:36:26.083732 27639 leader_election.cc:302] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71d4233cac234f918e9faa7384a1fbb9; no voters:
I20260619 20:36:26.083879 27639 leader_election.cc:288] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:26.083954 27644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:26.084131 27639 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:26.084146 27644 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9 [term 1 LEADER]: Becoming Leader. State: Replica: 71d4233cac234f918e9faa7384a1fbb9, State: Running, Role: LEADER
I20260619 20:36:26.084339 27644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71d4233cac234f918e9faa7384a1fbb9" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35573 } }
I20260619 20:36:26.084726 27646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71d4233cac234f918e9faa7384a1fbb9. Latest consensus state: current_term: 1 leader_uuid: "71d4233cac234f918e9faa7384a1fbb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71d4233cac234f918e9faa7384a1fbb9" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35573 } } }
I20260619 20:36:26.084728 27645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71d4233cac234f918e9faa7384a1fbb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71d4233cac234f918e9faa7384a1fbb9" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35573 } } }
I20260619 20:36:26.085000 27646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:26.085012 27645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:26.085976 27653 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:26.087098 27653 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:26.088925 27653 catalog_manager.cc:1347] Generated new cluster ID: d5164e404ec2443bacb6ce547b419867
I20260619 20:36:26.088982 27653 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:36:26 dist-test-slave-0t40 krb5kdc[27548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:26.094030 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.082327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60732 (local address 127.25.62.190:35573)
0619 20:36:26.082565 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.082574 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.082606 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:36:26.082729 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.082736 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.082790 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.082932 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:26.083674 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.085280 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.086367 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.086912 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.087034 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.089979 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:26.089983 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:26.089983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:26.089995 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:26.092138 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:26.092562 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:26.092565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:26.092565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:26.092711 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:26.092948 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:26.092948 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:26.092949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:26.093295 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:26.093470 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.093585 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.093831 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":123,"threads_started":1}
I20260619 20:36:26.111654 27653 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:26.112227 27653 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:26.121086 27653 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9: Generated new TSK 0
I20260619 20:36:26.121326 27653 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
I20260619 20:36:26.156991 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:35573
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:39141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:26.241438 27665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:26.241622 27665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:26.241645 27665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:26.242712 27665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:26.242748 27665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:26.242769 27665 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 20:36:26.242779 27665 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:36:26.242796 27665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:36:26.244135 27665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:39141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.62.190:35573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:26.244310 27665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:26.244470 27665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:26.249348 27673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:26.249336 27671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:26.249333 27670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:26.249619 27665 server_base.cc:1034] running on GCE node
I20260619 20:36:26.249792 27665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:26.250048 27665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:26.251245 27665 hybrid_clock.cc:648] HybridClock initialized: now 1781901386251238 us; error 40 us; skew 500 ppm
Jun 19 20:36:26 dist-test-slave-0t40 krb5kdc[27548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901386, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:26.253671 27665 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:36:26.254082 27665 webserver.cc:460] Webserver started at http://127.25.62.129:39603/ using document root <none> and password file <none>
I20260619 20:36:26.254369 27665 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:26.254415 27665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:26.254561 27665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:26.255790 27665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "6784121b23d7441f88f6e0b048105cdf"
format_stamp: "Formatted at 2026-06-19 20:36:26 on dist-test-slave-0t40"
server_key: "53584c2cc84f0e98af9eafe184efd809"
server_key_iv: "277a361863a59cd2b1049278d1fca3d8"
server_key_version: "encryptionkey@0"
I20260619 20:36:26.256155 27665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "6784121b23d7441f88f6e0b048105cdf"
format_stamp: "Formatted at 2026-06-19 20:36:26 on dist-test-slave-0t40"
server_key: "53584c2cc84f0e98af9eafe184efd809"
server_key_iv: "277a361863a59cd2b1049278d1fca3d8"
server_key_version: "encryptionkey@0"
I20260619 20:36:26.257526 27665 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:36:26.258253 27680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:26.258502 27665 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:26.258580 27665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "6784121b23d7441f88f6e0b048105cdf"
format_stamp: "Formatted at 2026-06-19 20:36:26 on dist-test-slave-0t40"
server_key: "53584c2cc84f0e98af9eafe184efd809"
server_key_iv: "277a361863a59cd2b1049278d1fca3d8"
server_key_version: "encryptionkey@0"
I20260619 20:36:26.258679 27665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:26.265684 27665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:26.267498 27665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:26.267642 27665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:26.268239 27665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:26.268291 27665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:26.268324 27665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:26.268343 27665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 20:36:26 dist-test-slave-0t40 krb5kdc[27548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901386, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:26.275436 27665 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:38485
I20260619 20:36:26.275825 27665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:36:26.276228 27795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:38485 every 8 connection(s)
I20260619 20:36:26.278743 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.268664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:45145 (local address 127.25.62.190:35573)
0619 20:36:26.268737 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.268739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.268911 (+ 172us) server_negotiation.cc:407] Connection header received
0619 20:36:26.269439 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.269441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.269458 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.269551 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:26.270164 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.270990 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.272712 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.272940 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.273039 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.275912 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:26.275913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:26.275914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:26.275922 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:26.277333 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:26.277743 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:26.277744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:26.277744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:26.277802 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:26.278089 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:26.278090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:26.278091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:26.278213 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:26.278292 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.278568 (+ 276us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.278652 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 20:36:26.278928 27689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.268622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:35573 (local address 127.25.62.129:45145)
0619 20:36:26.268773 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:26.268811 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:26.269336 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:26.269591 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:26.269595 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:26.269822 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:26.270066 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.270070 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.271124 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.271125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.272611 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.272615 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.273084 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.273085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.273211 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.274120 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:26.274128 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:26.275781 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:26.277418 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:26.277425 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:26.277427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:26.277654 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:26.277891 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:26.277892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:26.277893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:26.278004 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:26.278283 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:26.278289 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:26.278429 (+ 140us) client_negotiation.cc:769] Sending connection context
0619 20:36:26.278551 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 20:36:26.278706 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260619 20:36:26.279479 27796 heartbeater.cc:347] Connected to a master server at 127.25.62.190:35573
I20260619 20:36:26.279592 27796 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:26.279793 27796 heartbeater.cc:511] Master 127.25.62.190:35573 requested a full tablet report, sending...
I20260619 20:36:26.280428 27592 ts_manager.cc:194] Registered new tserver with Master: 6784121b23d7441f88f6e0b048105cdf (127.25.62.129:38485)
I20260619 20:36:26.281325 27592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:45145
I20260619 20:36:26.281895 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 27665
I20260619 20:36:26.281980 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:36:26.282161 25850 external_mini_cluster.cc:1428] Setting key 79726606e26524b285b485cbaec5f223
I20260619 20:36:26.287331 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.280615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:44767 (local address 127.25.62.190:35573)
0619 20:36:26.280681 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.280683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.280693 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:36:26.280714 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.280715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.280724 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.280789 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:26.281109 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.281758 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.282690 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.282970 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.283026 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.284145 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:26.284146 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:26.284147 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:26.284154 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:26.286010 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:26.286440 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:26.286441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:26.286441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:26.286505 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:26.286736 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:26.286736 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:26.286737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:26.286846 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:26.286938 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.287193 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.287266 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 20:36:26.287400 27802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.280494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:35573 (local address 127.25.62.129:44767)
0619 20:36:26.280610 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:26.280621 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:26.280669 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:26.280807 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:26.280807 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:26.280945 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:26.281001 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.281002 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.281867 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.281868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.282610 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.282612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.283067 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.283068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.283150 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.283542 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:26.283548 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:26.283988 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:26.286099 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:26.286100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:26.286101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:26.286336 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:26.286590 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:26.286591 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:26.286592 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:26.286664 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:26.287005 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:26.287006 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:26.287066 (+ 60us) client_negotiation.cc:769] Sending connection context
0619 20:36:26.287160 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 20:36:26.287281 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":4,"thread_start_us":66,"threads_started":1}
WARNING: no policy specified for kudu/127.25.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
I20260619 20:36:26.308647 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:35573
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:39141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:26.390686 27805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:26.390877 27805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:26.390900 27805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:26.391947 27805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:26.391979 27805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:26.391999 27805 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 20:36:26.392010 27805 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:36:26.392027 27805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:36:26.393535 27805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:39141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.62.190:35573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:26.393713 27805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:26.393870 27805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:26.398720 27811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:26.398706 27810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:26.398845 27805 server_base.cc:1034] running on GCE node
W20260619 20:36:26.398707 27813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:26.399127 27805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:26.399374 27805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:26.400548 27805 hybrid_clock.cc:648] HybridClock initialized: now 1781901386400535 us; error 52 us; skew 500 ppm
Jun 19 20:36:26 dist-test-slave-0t40 krb5kdc[27548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901386, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:26.403131 27805 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:36:26.403553 27805 webserver.cc:460] Webserver started at http://127.25.62.130:34389/ using document root <none> and password file <none>
I20260619 20:36:26.403775 27805 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:26.403847 27805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:26.404011 27805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:26.405092 27805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "eaefcaa82f5e45039d58b6493a21ba89"
format_stamp: "Formatted at 2026-06-19 20:36:26 on dist-test-slave-0t40"
server_key: "31ec501253d9ef35df63afdb42b633e5"
server_key_iv: "7f541fdcdf860138b516b40c3383f872"
server_key_version: "encryptionkey@0"
I20260619 20:36:26.405488 27805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "eaefcaa82f5e45039d58b6493a21ba89"
format_stamp: "Formatted at 2026-06-19 20:36:26 on dist-test-slave-0t40"
server_key: "31ec501253d9ef35df63afdb42b633e5"
server_key_iv: "7f541fdcdf860138b516b40c3383f872"
server_key_version: "encryptionkey@0"
I20260619 20:36:26.406847 27805 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:26.407641 27820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:26.407823 27805 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:26.407946 27805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "eaefcaa82f5e45039d58b6493a21ba89"
format_stamp: "Formatted at 2026-06-19 20:36:26 on dist-test-slave-0t40"
server_key: "31ec501253d9ef35df63afdb42b633e5"
server_key_iv: "7f541fdcdf860138b516b40c3383f872"
server_key_version: "encryptionkey@0"
I20260619 20:36:26.408026 27805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:26.427608 27805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:26.430272 27805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:26.430461 27805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:26.431059 27805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:26.431136 27805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:26.431180 27805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:26.431210 27805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 20:36:26 dist-test-slave-0t40 krb5kdc[27548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901386, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:26.439332 27805 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:39075
I20260619 20:36:26.439643 27937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:39075 every 8 connection(s)
I20260619 20:36:26.439710 27805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:36:26.441903 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.431634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:46221 (local address 127.25.62.190:35573)
0619 20:36:26.431720 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.431723 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.431876 (+ 153us) server_negotiation.cc:407] Connection header received
0619 20:36:26.432391 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.432394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.432410 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.432506 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:26.433439 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.434258 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.435852 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.436182 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.436248 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.439085 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:26.439087 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:26.439087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:26.439096 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:26.440624 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:26.441019 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:26.441019 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:26.441019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:26.441115 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:26.441368 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:26.441368 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:26.441368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:26.441490 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:26.441542 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.441784 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.441843 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 20:36:26.442068 27829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.431555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:35573 (local address 127.25.62.130:46221)
0619 20:36:26.431739 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:26.431783 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:26.432282 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:26.432558 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:26.432561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:26.432986 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:26.433319 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.433327 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.434384 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.434384 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.435754 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.435757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.436324 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.436325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.436488 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.437105 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:26.437115 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:26.438954 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:26.440698 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:26.440702 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:26.440704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:26.440942 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:26.441189 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:26.441190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:26.441190 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:26.441302 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:26.441550 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:26.441554 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:26.441666 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 20:36:26.441753 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 20:36:26.441871 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260619 20:36:26.442587 27938 heartbeater.cc:347] Connected to a master server at 127.25.62.190:35573
I20260619 20:36:26.442673 27938 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:26.442871 27938 heartbeater.cc:511] Master 127.25.62.190:35573 requested a full tablet report, sending...
I20260619 20:36:26.443286 27592 ts_manager.cc:194] Registered new tserver with Master: eaefcaa82f5e45039d58b6493a21ba89 (127.25.62.130:39075)
I20260619 20:36:26.443593 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 27805
I20260619 20:36:26.443696 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:36:26.443781 27592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:46221
I20260619 20:36:26.443934 25850 external_mini_cluster.cc:1428] Setting key 1bc67a3879f3c51ff54985f1689c19cf
I20260619 20:36:26.450173 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.443741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:59137 (local address 127.25.62.190:35573)
0619 20:36:26.443783 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.443785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.443965 (+ 180us) server_negotiation.cc:407] Connection header received
0619 20:36:26.444062 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.444063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.444074 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.444144 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:26.444491 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.445117 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.445768 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.446084 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.446164 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.447284 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:26.447285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:26.447285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:26.447291 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:26.448879 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:26.449340 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:26.449341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:26.449342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:26.449402 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:26.449630 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:26.449631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:26.449631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:26.449774 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:26.449825 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.450034 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.450098 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:36:26.450309 27944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.443738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:35573 (local address 127.25.62.130:59137)
0619 20:36:26.443905 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:26.443914 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:26.443989 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:26.444149 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:26.444150 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:26.444334 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:26.444405 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.444406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.445194 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.445194 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.445682 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.445683 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.446173 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.446174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.446257 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.446770 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:26.446777 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:26.447177 (+ 400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:26.448985 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:26.448987 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:26.448988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:26.449244 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:26.449481 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:26.449483 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:26.449483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:26.449558 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:26.449849 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:26.449850 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:26.449920 (+ 70us) client_negotiation.cc:769] Sending connection context
0619 20:36:26.450018 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 20:36:26.450148 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for kudu/127.25.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
I20260619 20:36:26.473304 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:35573
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:39141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:26.553642 27947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:26.553848 27947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:26.553871 27947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:26.555060 27947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:26.555099 27947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:26.555120 27947 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 20:36:26.555132 27947 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:36:26.555148 27947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:36:26.556663 27947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:39141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.62.190:35573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:26.556924 27947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:26.557189 27947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:26.562299 27952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:26.562299 27955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:26.562331 27953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:26.562752 27947 server_base.cc:1034] running on GCE node
I20260619 20:36:26.562971 27947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:26.563238 27947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:26.564420 27947 hybrid_clock.cc:648] HybridClock initialized: now 1781901386564403 us; error 34 us; skew 500 ppm
Jun 19 20:36:26 dist-test-slave-0t40 krb5kdc[27548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901386, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:26.566905 27947 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:36:26.567397 27947 webserver.cc:460] Webserver started at http://127.25.62.131:38259/ using document root <none> and password file <none>
I20260619 20:36:26.567647 27947 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:26.567721 27947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:26.567875 27947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:26.568878 27947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "047ab3185fee4b09820be6c918bfa681"
format_stamp: "Formatted at 2026-06-19 20:36:26 on dist-test-slave-0t40"
server_key: "65389c875e3ccd561e46587d79116d0a"
server_key_iv: "e85f9b565843c3f8ad9053dffd9fa7cb"
server_key_version: "encryptionkey@0"
I20260619 20:36:26.569269 27947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "047ab3185fee4b09820be6c918bfa681"
format_stamp: "Formatted at 2026-06-19 20:36:26 on dist-test-slave-0t40"
server_key: "65389c875e3ccd561e46587d79116d0a"
server_key_iv: "e85f9b565843c3f8ad9053dffd9fa7cb"
server_key_version: "encryptionkey@0"
I20260619 20:36:26.570556 27947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:26.571353 27962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:26.571553 27947 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:26.571666 27947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "047ab3185fee4b09820be6c918bfa681"
format_stamp: "Formatted at 2026-06-19 20:36:26 on dist-test-slave-0t40"
server_key: "65389c875e3ccd561e46587d79116d0a"
server_key_iv: "e85f9b565843c3f8ad9053dffd9fa7cb"
server_key_version: "encryptionkey@0"
I20260619 20:36:26.571756 27947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:26.583006 27947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:26.585219 27947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:26.585395 27947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:26.586045 27947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:26.586138 27947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:26.586184 27947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:26.586210 27947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 20:36:26 dist-test-slave-0t40 krb5kdc[27548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901386, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:26.593343 27947 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:44623
I20260619 20:36:26.593432 28078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:44623 every 8 connection(s)
I20260619 20:36:26.593755 27947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:36:26.596078 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.586372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:59525 (local address 127.25.62.190:35573)
0619 20:36:26.586416 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.586418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.586694 (+ 276us) server_negotiation.cc:407] Connection header received
0619 20:36:26.587205 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.587207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.587220 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.587306 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:26.587918 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.588718 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.590417 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.590631 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.590686 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.593259 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:26.593260 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:26.593261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:26.593269 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:26.594866 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:26.595297 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:26.595298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:26.595298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:26.595350 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:26.595578 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:26.595579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:26.595579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:26.595681 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:26.595732 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.595941 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.596016 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:36:26.596269 27971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.586378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:35573 (local address 127.25.62.131:59525)
0619 20:36:26.586543 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:26.586582 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:26.587087 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:26.587352 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:26.587355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:26.587568 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:26.587806 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.587813 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.588856 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.588858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.590318 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.590320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.590704 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.590705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.590864 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.591446 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:26.591465 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:26.593183 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:26.594943 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:26.594947 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:26.594948 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:26.595206 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:26.595419 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:26.595420 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:26.595421 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:26.595515 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:26.595740 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:26.595744 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:26.595849 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 20:36:26.595926 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 20:36:26.596039 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":105,"threads_started":1}
I20260619 20:36:26.596805 28079 heartbeater.cc:347] Connected to a master server at 127.25.62.190:35573
I20260619 20:36:26.596879 28079 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:26.597074 28079 heartbeater.cc:511] Master 127.25.62.190:35573 requested a full tablet report, sending...
I20260619 20:36:26.597443 27593 ts_manager.cc:194] Registered new tserver with Master: 047ab3185fee4b09820be6c918bfa681 (127.25.62.131:44623)
I20260619 20:36:26.597972 27593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:59525
I20260619 20:36:26.598237 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 27947
I20260619 20:36:26.598320 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:36:26.598497 25850 external_mini_cluster.cc:1428] Setting key 4f12b6ad7416e77c346c7257533b4720
I20260619 20:36:26.599246 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:36:26.603933 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.597898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:50759 (local address 127.25.62.190:35573)
0619 20:36:26.597935 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.597936 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.597972 (+ 36us) server_negotiation.cc:407] Connection header received
0619 20:36:26.598062 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.598063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.598073 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.598129 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:26.598519 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.599238 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.599940 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.600152 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.600231 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.601293 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:26.601295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:26.601295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:26.601302 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:26.602944 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:26.603248 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:26.603249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:26.603250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:26.603309 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:26.603461 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:26.603461 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:26.603462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:26.603584 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:26.603660 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.603767 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.603844 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:36:26.603943 28085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.597790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:35573 (local address 127.25.62.131:50759)
0619 20:36:26.597918 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:26.597928 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:26.597986 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:26.598147 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:26.598148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:26.598366 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:26.598430 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.598432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.599314 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.599315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.599847 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.599849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.600241 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.600242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.600314 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.600774 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:26.600779 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:26.601199 (+ 420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:26.603028 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:26.603029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:26.603029 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:26.603183 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:26.603375 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:26.603376 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:26.603376 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:26.603417 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:26.603659 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:26.603660 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:26.603693 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 20:36:26.603744 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 20:36:26.603857 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
Jun 19 20:36:26 dist-test-slave-0t40 krb5kdc[27548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 20:36:26 dist-test-slave-0t40 krb5kdc[27548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:26.623015 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.615774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60742 (local address 127.25.62.190:35573)
0619 20:36:26.615835 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.615836 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.615866 (+ 30us) server_negotiation.cc:407] Connection header received
0619 20:36:26.615946 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.615947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.615955 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.616003 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:26.616578 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.617123 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.617971 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.618161 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.618224 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.620607 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:26.620608 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:26.620608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:26.620614 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:26.622138 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:26.622446 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:26.622447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:26.622447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:26.622496 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:26.622645 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:26.622645 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:26.622645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:26.622729 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:26.622795 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.622884 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.622954 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:36:26.624589 27593 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:36:26.626214 27593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:36:26.632781 28097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.628813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:38485 (local address 127.0.0.1:56306)
0619 20:36:26.629101 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:26.629109 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:26.629150 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:26.629474 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:26.629475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:26.629477 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:26.629541 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.629543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.630417 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.630418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.631608 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.631611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.632489 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.632490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.632565 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.632579 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 20:36:26.632661 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 20:36:26.632713 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":1,"thread_start_us":68,"threads_started":1}
I20260619 20:36:26.632839 28101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.629031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56306 (local address 127.25.62.129:38485)
0619 20:36:26.629263 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.629267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.629277 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:36:26.629315 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.629318 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.629333 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.629408 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:26.629623 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.630315 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.631679 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.632380 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.632469 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.632526 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.632694 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.632736 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":86,"threads_started":1}
I20260619 20:36:26.633147 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.628749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38222 (local address 127.25.62.130:39075)
0619 20:36:26.629064 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.629069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.629082 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:36:26.629368 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.629371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.629387 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.629436 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:26.629643 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.630711 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.631910 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.632606 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.632704 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.632758 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.632979 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.633023 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":88,"threads_started":1}
I20260619 20:36:26.633339 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.628625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:39075 (local address 127.0.0.1:38222)
0619 20:36:26.628950 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:26.629001 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:26.629248 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:26.629517 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:26.629518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:26.629519 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:26.629548 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.629549 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.630807 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.630808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.631824 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.631826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.632793 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.632794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.632866 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.632870 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:26.633225 (+ 355us) client_negotiation.cc:240] Negotiation successful
0619 20:36:26.633271 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":74,"threads_started":1}
I20260619 20:36:26.633637 27730 tablet_service.cc:1467] Processing CreateTablet for tablet bc5b7c4d755544208de357007ce04e4a (DEFAULT_TABLE table=test-table [id=0d4733556bcd4463a55488bf31c4f406]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:26.634049 27730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc5b7c4d755544208de357007ce04e4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:26.634191 28100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.628622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:44623 (local address 127.0.0.1:55438)
0619 20:36:26.628951 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:26.628998 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:26.629142 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:26.629421 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:26.629424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:26.629426 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:26.629490 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.629491 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.631893 (+ 2402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.631893 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.632604 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.632606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.633901 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.633902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.634004 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.634008 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:26.634078 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 20:36:26.634118 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":49,"threads_started":1}
I20260619 20:36:26.634249 28098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.628697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55438 (local address 127.25.62.131:44623)
0619 20:36:26.628881 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.628887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.629094 (+ 207us) server_negotiation.cc:407] Connection header received
0619 20:36:26.629262 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.629265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.629280 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.629349 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:26.629569 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.630711 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.632952 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.633789 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.633862 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.633917 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.634088 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.634154 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":120,"threads_started":1}
I20260619 20:36:26.634876 28013 tablet_service.cc:1467] Processing CreateTablet for tablet bc5b7c4d755544208de357007ce04e4a (DEFAULT_TABLE table=test-table [id=0d4733556bcd4463a55488bf31c4f406]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:26.635193 28013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc5b7c4d755544208de357007ce04e4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:26.637800 28102 tablet_bootstrap.cc:492] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf: Bootstrap starting.
I20260619 20:36:26.637705 27870 tablet_service.cc:1467] Processing CreateTablet for tablet bc5b7c4d755544208de357007ce04e4a (DEFAULT_TABLE table=test-table [id=0d4733556bcd4463a55488bf31c4f406]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:26.638041 27870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc5b7c4d755544208de357007ce04e4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:26.638407 28103 tablet_bootstrap.cc:492] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681: Bootstrap starting.
I20260619 20:36:26.638442 28102 tablet_bootstrap.cc:654] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:26.638767 28102 log.cc:824] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:26.639245 28103 tablet_bootstrap.cc:654] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:26.639542 28103 log.cc:824] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:26.639720 28102 tablet_bootstrap.cc:492] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf: No bootstrap required, opened a new log
I20260619 20:36:26.639781 28102 ts_tablet_manager.cc:1397] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:36:26.640182 28103 tablet_bootstrap.cc:492] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681: No bootstrap required, opened a new log
I20260619 20:36:26.640260 28103 ts_tablet_manager.cc:1397] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:36:26.641726 28106 tablet_bootstrap.cc:492] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89: Bootstrap starting.
I20260619 20:36:26.641796 28103 raft_consensus.cc:348] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } }
I20260619 20:36:26.641953 28103 raft_consensus.cc:374] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:26.641993 28103 raft_consensus.cc:729] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047ab3185fee4b09820be6c918bfa681, State: Initialized, Role: FOLLOWER
I20260619 20:36:26.642123 28103 consensus_queue.cc:260] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } }
I20260619 20:36:26.642340 28106 tablet_bootstrap.cc:654] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:26.642380 28103 ts_tablet_manager.cc:1428] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:36:26.642445 28079 heartbeater.cc:503] Master 127.25.62.190:35573 was elected leader, sending a full tablet report...
I20260619 20:36:26.642335 28102 raft_consensus.cc:348] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } }
I20260619 20:36:26.642494 28102 raft_consensus.cc:374] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:26.642530 28102 raft_consensus.cc:729] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6784121b23d7441f88f6e0b048105cdf, State: Initialized, Role: FOLLOWER
I20260619 20:36:26.642668 28102 consensus_queue.cc:260] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } }
I20260619 20:36:26.642724 28106 log.cc:824] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:26.643275 27796 heartbeater.cc:503] Master 127.25.62.190:35573 was elected leader, sending a full tablet report...
I20260619 20:36:26.643275 28102 ts_tablet_manager.cc:1428] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:36:26.643589 28106 tablet_bootstrap.cc:492] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89: No bootstrap required, opened a new log
I20260619 20:36:26.643659 28106 ts_tablet_manager.cc:1397] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:36:26.645328 28106 raft_consensus.cc:348] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } }
I20260619 20:36:26.645470 28106 raft_consensus.cc:374] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:26.645509 28106 raft_consensus.cc:729] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaefcaa82f5e45039d58b6493a21ba89, State: Initialized, Role: FOLLOWER
I20260619 20:36:26.645619 28106 consensus_queue.cc:260] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } }
I20260619 20:36:26.645855 28106 ts_tablet_manager.cc:1428] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:36:26.645937 27938 heartbeater.cc:503] Master 127.25.62.190:35573 was elected leader, sending a full tablet report...
W20260619 20:36:26.690435 27939 tablet.cc:2369] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:36:26.776572 27797 tablet.cc:2369] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:36:26.844549 28080 tablet.cc:2369] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:36:26.874902 28107 raft_consensus.cc:482] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:36:26.875029 28107 raft_consensus.cc:504] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } }
I20260619 20:36:26.875458 28107 leader_election.cc:288] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eaefcaa82f5e45039d58b6493a21ba89 (127.25.62.130:39075), 6784121b23d7441f88f6e0b048105cdf (127.25.62.129:38485)
I20260619 20:36:26.878506 28111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.875775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:38485 (local address 127.25.62.131:32955)
0619 20:36:26.875948 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:26.875957 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:26.876008 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:26.876171 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:26.876172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:26.876173 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:26.876246 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.876248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.876899 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.876900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.877678 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.877680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.878240 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.878241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.878316 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.878321 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:36:26.878377 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 20:36:26.878415 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":86,"threads_started":1}
I20260619 20:36:26.878525 28101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.875860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:32955 (local address 127.25.62.129:38485)
0619 20:36:26.875903 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.875907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.876052 (+ 145us) server_negotiation.cc:407] Connection header received
0619 20:36:26.876079 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.876080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.876090 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.876144 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:26.876322 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.876813 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.877769 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.878150 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.878207 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.878214 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.878402 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.878443 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:36:26.878660 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.875751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:54175 (local address 127.25.62.130:39075)
0619 20:36:26.875802 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.875804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.875815 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:36:26.875886 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.875887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.875897 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.875949 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:26.876171 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.876713 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.877834 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.878236 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.878310 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.878317 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.878533 (+ 216us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.878576 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:36:26.878614 27971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.875613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:39075 (local address 127.25.62.131:54175)
0619 20:36:26.875731 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:26.875745 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:26.875814 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:26.875977 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:26.875978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:26.875980 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:26.876063 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.876065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.876795 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.876796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.877739 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.877742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.878332 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.878333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.878443 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.878446 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:36:26.878504 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 20:36:26.878537 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":105}
I20260619 20:36:26.879180 27887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc5b7c4d755544208de357007ce04e4a" candidate_uuid: "047ab3185fee4b09820be6c918bfa681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaefcaa82f5e45039d58b6493a21ba89" is_pre_election: true
I20260619 20:36:26.879200 27750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc5b7c4d755544208de357007ce04e4a" candidate_uuid: "047ab3185fee4b09820be6c918bfa681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6784121b23d7441f88f6e0b048105cdf" is_pre_election: true
I20260619 20:36:26.879316 27887 raft_consensus.cc:2454] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 047ab3185fee4b09820be6c918bfa681 in term 0.
I20260619 20:36:26.879362 27750 raft_consensus.cc:2454] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 047ab3185fee4b09820be6c918bfa681 in term 0.
I20260619 20:36:26.879689 27964 leader_election.cc:302] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047ab3185fee4b09820be6c918bfa681, 6784121b23d7441f88f6e0b048105cdf; no voters:
I20260619 20:36:26.879832 28107 raft_consensus.cc:2781] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:36:26.879889 28107 raft_consensus.cc:482] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:36:26.879913 28107 raft_consensus.cc:3037] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:26.880645 28107 raft_consensus.cc:504] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } }
I20260619 20:36:26.880795 28107 leader_election.cc:288] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [CANDIDATE]: Term 1 election: Requested vote from peers eaefcaa82f5e45039d58b6493a21ba89 (127.25.62.130:39075), 6784121b23d7441f88f6e0b048105cdf (127.25.62.129:38485)
I20260619 20:36:26.880941 27887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc5b7c4d755544208de357007ce04e4a" candidate_uuid: "047ab3185fee4b09820be6c918bfa681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaefcaa82f5e45039d58b6493a21ba89"
I20260619 20:36:26.880975 27750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc5b7c4d755544208de357007ce04e4a" candidate_uuid: "047ab3185fee4b09820be6c918bfa681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6784121b23d7441f88f6e0b048105cdf"
I20260619 20:36:26.881026 27887 raft_consensus.cc:3037] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:26.881096 27750 raft_consensus.cc:3037] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:26.881762 27887 raft_consensus.cc:2454] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 047ab3185fee4b09820be6c918bfa681 in term 1.
I20260619 20:36:26.881780 27750 raft_consensus.cc:2454] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 047ab3185fee4b09820be6c918bfa681 in term 1.
I20260619 20:36:26.881930 27964 leader_election.cc:302] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047ab3185fee4b09820be6c918bfa681, eaefcaa82f5e45039d58b6493a21ba89; no voters:
I20260619 20:36:26.882010 28107 raft_consensus.cc:2781] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:26.882153 28107 raft_consensus.cc:686] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [term 1 LEADER]: Becoming Leader. State: Replica: 047ab3185fee4b09820be6c918bfa681, State: Running, Role: LEADER
I20260619 20:36:26.882261 28107 consensus_queue.cc:237] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } }
I20260619 20:36:26.883000 27592 catalog_manager.cc:5495] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 reported cstate change: term changed from 0 to 1, leader changed from <none> to 047ab3185fee4b09820be6c918bfa681 (127.25.62.131). New cstate: current_term: 1 leader_uuid: "047ab3185fee4b09820be6c918bfa681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } health_report { overall_health: UNKNOWN } } }
Jun 19 20:36:26 dist-test-slave-0t40 krb5kdc[27548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901386, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:26.932911 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.925858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60756 (local address 127.25.62.190:35573)
0619 20:36:26.925937 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.925939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.926065 (+ 126us) server_negotiation.cc:407] Connection header received
0619 20:36:26.926179 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.926180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.926198 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.926264 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:26.926731 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.927227 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.927930 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.928117 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.928210 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.930340 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:26.930342 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:26.930342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:26.930349 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:26.931669 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:26.932064 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:26.932065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:26.932066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:26.932124 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:26.932330 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:26.932331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:26.932331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:26.932451 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:26.932533 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.932694 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.932781 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 20:36:26.933009 28119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.925802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:35573 (local address 127.0.0.1:60756)
0619 20:36:26.925987 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:26.925996 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:26.926087 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:26.926268 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:26.926269 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:26.926595 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:26.926641 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.926644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.927315 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.927316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.927840 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.927842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.928211 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.928212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.928273 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.928783 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:26.928787 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:26.930244 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:26.931774 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:26.931782 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:26.931784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:26.931982 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:26.932196 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:26.932196 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:26.932197 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:26.932261 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:26.932525 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:26.932533 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:26.932577 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 20:36:26.932673 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 20:36:26.932864 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":93,"thread_start_us":85,"threads_started":1}
I20260619 20:36:26.935031 27592 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:60756:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 20:36:26.935247 27592 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:36:26.936816 27870 tablet_service.cc:1467] Processing CreateTablet for tablet 064191579c994edc96f03fd413d22380 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cc3d341d72c145ba87add39fe7d9cbeb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:36:26.936861 27730 tablet_service.cc:1467] Processing CreateTablet for tablet 064191579c994edc96f03fd413d22380 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cc3d341d72c145ba87add39fe7d9cbeb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:36:26.937023 27870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 064191579c994edc96f03fd413d22380. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:26.937073 27730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 064191579c994edc96f03fd413d22380. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:26.937106 28013 tablet_service.cc:1467] Processing CreateTablet for tablet 064191579c994edc96f03fd413d22380 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cc3d341d72c145ba87add39fe7d9cbeb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:36:26.937238 28013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 064191579c994edc96f03fd413d22380. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:26.938249 28106 tablet_bootstrap.cc:492] T 064191579c994edc96f03fd413d22380 P eaefcaa82f5e45039d58b6493a21ba89: Bootstrap starting.
I20260619 20:36:26.938658 28102 tablet_bootstrap.cc:492] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf: Bootstrap starting.
I20260619 20:36:26.938680 28103 tablet_bootstrap.cc:492] T 064191579c994edc96f03fd413d22380 P 047ab3185fee4b09820be6c918bfa681: Bootstrap starting.
I20260619 20:36:26.938949 28106 tablet_bootstrap.cc:654] T 064191579c994edc96f03fd413d22380 P eaefcaa82f5e45039d58b6493a21ba89: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:26.939020 28102 tablet_bootstrap.cc:654] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:26.939112 28103 tablet_bootstrap.cc:654] T 064191579c994edc96f03fd413d22380 P 047ab3185fee4b09820be6c918bfa681: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:26.939632 28102 tablet_bootstrap.cc:492] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf: No bootstrap required, opened a new log
I20260619 20:36:26.939716 28102 ts_tablet_manager.cc:1397] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:26.939831 28103 tablet_bootstrap.cc:492] T 064191579c994edc96f03fd413d22380 P 047ab3185fee4b09820be6c918bfa681: No bootstrap required, opened a new log
I20260619 20:36:26.939877 28103 ts_tablet_manager.cc:1397] T 064191579c994edc96f03fd413d22380 P 047ab3185fee4b09820be6c918bfa681: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:26.939870 28102 raft_consensus.cc:348] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:26.939960 28102 raft_consensus.cc:374] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:26.939990 28102 raft_consensus.cc:729] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6784121b23d7441f88f6e0b048105cdf, State: Initialized, Role: FOLLOWER
I20260619 20:36:26.940040 28102 consensus_queue.cc:260] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:26.940029 28103 raft_consensus.cc:348] T 064191579c994edc96f03fd413d22380 P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:26.940099 28103 raft_consensus.cc:374] T 064191579c994edc96f03fd413d22380 P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:26.940130 28103 raft_consensus.cc:729] T 064191579c994edc96f03fd413d22380 P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047ab3185fee4b09820be6c918bfa681, State: Initialized, Role: FOLLOWER
I20260619 20:36:26.940150 28102 ts_tablet_manager.cc:1428] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:26.940178 28103 consensus_queue.cc:260] T 064191579c994edc96f03fd413d22380 P 047ab3185fee4b09820be6c918bfa681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:26.940275 28103 ts_tablet_manager.cc:1428] T 064191579c994edc96f03fd413d22380 P 047ab3185fee4b09820be6c918bfa681: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:26.940227 28109 tablet_replica.cc:439] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } } }
I20260619 20:36:26.940321 28109 tablet_replica.cc:442] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:26.940310 28107 tablet_replica.cc:439] T 064191579c994edc96f03fd413d22380 P 047ab3185fee4b09820be6c918bfa681: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } } }
I20260619 20:36:26.940369 28107 tablet_replica.cc:442] T 064191579c994edc96f03fd413d22380 P 047ab3185fee4b09820be6c918bfa681: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:26.941228 28106 tablet_bootstrap.cc:492] T 064191579c994edc96f03fd413d22380 P eaefcaa82f5e45039d58b6493a21ba89: No bootstrap required, opened a new log
I20260619 20:36:26.941274 28106 ts_tablet_manager.cc:1397] T 064191579c994edc96f03fd413d22380 P eaefcaa82f5e45039d58b6493a21ba89: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 20:36:26.941448 28106 raft_consensus.cc:348] T 064191579c994edc96f03fd413d22380 P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:26.941510 28106 raft_consensus.cc:374] T 064191579c994edc96f03fd413d22380 P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:26.941558 28106 raft_consensus.cc:729] T 064191579c994edc96f03fd413d22380 P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaefcaa82f5e45039d58b6493a21ba89, State: Initialized, Role: FOLLOWER
I20260619 20:36:26.941612 28106 consensus_queue.cc:260] T 064191579c994edc96f03fd413d22380 P eaefcaa82f5e45039d58b6493a21ba89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:26.941705 28106 ts_tablet_manager.cc:1428] T 064191579c994edc96f03fd413d22380 P eaefcaa82f5e45039d58b6493a21ba89: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:26.941783 28110 tablet_replica.cc:439] T 064191579c994edc96f03fd413d22380 P eaefcaa82f5e45039d58b6493a21ba89: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } } }
I20260619 20:36:26.941851 28110 tablet_replica.cc:442] T 064191579c994edc96f03fd413d22380 P eaefcaa82f5e45039d58b6493a21ba89: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:26.949138 28109 raft_consensus.cc:482] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:36:26.949216 28109 raft_consensus.cc:504] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:26.949569 28109 leader_election.cc:288] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eaefcaa82f5e45039d58b6493a21ba89 (127.25.62.130:39075), 047ab3185fee4b09820be6c918bfa681 (127.25.62.131:44623)
I20260619 20:36:26.952731 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.949800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:55575 (local address 127.25.62.130:39075)
0619 20:36:26.949845 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.949847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.949946 (+ 99us) server_negotiation.cc:407] Connection header received
0619 20:36:26.949994 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.949995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.950008 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.950062 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:26.950239 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.950910 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.951827 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.952325 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.952382 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.952389 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.952595 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.952640 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:36:26.952764 28124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.949748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:44623 (local address 127.25.62.129:59185)
0619 20:36:26.949980 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:26.949991 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:26.950046 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:26.950172 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:26.950173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:26.950174 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:26.950269 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.950270 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.950862 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.950863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.951736 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.951739 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.952535 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.952536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.952612 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.952615 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:36:26.952665 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 20:36:26.952693 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":17,"thread_start_us":53,"threads_started":1}
I20260619 20:36:26.952797 28098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.949786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:59185 (local address 127.25.62.131:44623)
0619 20:36:26.949831 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.949833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.950033 (+ 200us) server_negotiation.cc:407] Connection header received
0619 20:36:26.950091 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.950092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.950101 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.950159 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:26.950342 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.950784 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.951859 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.952429 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.952497 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.952505 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.952690 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.952728 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:36:26.952919 28123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.949698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:39075 (local address 127.25.62.129:55575)
0619 20:36:26.949861 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:26.949870 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:26.949934 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:26.950085 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:26.950086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:26.950088 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:26.950163 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.950164 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.950999 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.951000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.951734 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.951736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.952424 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.952426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.952498 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.952502 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:26.952586 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 20:36:26.952629 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":3,"thread_start_us":65,"threads_started":1}
I20260619 20:36:26.953074 28033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064191579c994edc96f03fd413d22380" candidate_uuid: "6784121b23d7441f88f6e0b048105cdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047ab3185fee4b09820be6c918bfa681" is_pre_election: true
I20260619 20:36:26.953203 28033 raft_consensus.cc:2454] T 064191579c994edc96f03fd413d22380 P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6784121b23d7441f88f6e0b048105cdf in term 0.
I20260619 20:36:26.953395 27684 leader_election.cc:302] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047ab3185fee4b09820be6c918bfa681, 6784121b23d7441f88f6e0b048105cdf; no voters:
I20260619 20:36:26.953455 27887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064191579c994edc96f03fd413d22380" candidate_uuid: "6784121b23d7441f88f6e0b048105cdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaefcaa82f5e45039d58b6493a21ba89" is_pre_election: true
I20260619 20:36:26.953542 27887 raft_consensus.cc:2454] T 064191579c994edc96f03fd413d22380 P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6784121b23d7441f88f6e0b048105cdf in term 0.
I20260619 20:36:26.953541 28109 raft_consensus.cc:2781] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:36:26.953574 28109 raft_consensus.cc:482] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:36:26.953595 28109 raft_consensus.cc:3037] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:26.954272 28109 raft_consensus.cc:504] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:26.954411 28109 leader_election.cc:288] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf [CANDIDATE]: Term 1 election: Requested vote from peers eaefcaa82f5e45039d58b6493a21ba89 (127.25.62.130:39075), 047ab3185fee4b09820be6c918bfa681 (127.25.62.131:44623)
I20260619 20:36:26.954532 27887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064191579c994edc96f03fd413d22380" candidate_uuid: "6784121b23d7441f88f6e0b048105cdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaefcaa82f5e45039d58b6493a21ba89"
I20260619 20:36:26.954602 27887 raft_consensus.cc:3037] T 064191579c994edc96f03fd413d22380 P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:26.954559 28033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064191579c994edc96f03fd413d22380" candidate_uuid: "6784121b23d7441f88f6e0b048105cdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047ab3185fee4b09820be6c918bfa681"
I20260619 20:36:26.954694 28033 raft_consensus.cc:3037] T 064191579c994edc96f03fd413d22380 P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:26.955344 27887 raft_consensus.cc:2454] T 064191579c994edc96f03fd413d22380 P eaefcaa82f5e45039d58b6493a21ba89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6784121b23d7441f88f6e0b048105cdf in term 1.
I20260619 20:36:26.955404 28033 raft_consensus.cc:2454] T 064191579c994edc96f03fd413d22380 P 047ab3185fee4b09820be6c918bfa681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6784121b23d7441f88f6e0b048105cdf in term 1.
I20260619 20:36:26.955519 27682 leader_election.cc:302] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6784121b23d7441f88f6e0b048105cdf, eaefcaa82f5e45039d58b6493a21ba89; no voters:
I20260619 20:36:26.955607 28109 raft_consensus.cc:2781] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:26.955744 28109 raft_consensus.cc:686] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf [term 1 LEADER]: Becoming Leader. State: Replica: 6784121b23d7441f88f6e0b048105cdf, State: Running, Role: LEADER
I20260619 20:36:26.955843 28109 consensus_queue.cc:237] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:26.956460 28125 tablet_replica.cc:439] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf: TxnStatusTablet state changed. Reason: New leader 6784121b23d7441f88f6e0b048105cdf. Latest consensus state: current_term: 1 leader_uuid: "6784121b23d7441f88f6e0b048105cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } } }
I20260619 20:36:26.956569 28125 tablet_replica.cc:442] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:36:26.956631 27592 catalog_manager.cc:5495] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf reported cstate change: term changed from 0 to 1, leader changed from <none> to 6784121b23d7441f88f6e0b048105cdf (127.25.62.129). New cstate: current_term: 1 leader_uuid: "6784121b23d7441f88f6e0b048105cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } health_report { overall_health: UNKNOWN } } }
I20260619 20:36:26.956732 28127 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 20:36:26.956846 28127 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 20:36:26.966789 28119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.964180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:38485 (local address 127.0.0.1:56314)
0619 20:36:26.964258 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:26.964271 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:26.964321 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:26.964517 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:26.964518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:26.964519 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:36:26.964568 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.964570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.965263 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.965265 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.965846 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.965848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.966244 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.966244 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.966311 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.966331 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:36:26.966616 (+ 285us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:36:26.966626 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 20:36:26.966675 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 20:36:26.966721 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 20:36:26.966776 28101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.964246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56314 (local address 127.25.62.129:38485)
0619 20:36:26.964291 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.964294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.964302 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:26.964390 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.964391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.964408 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.964508 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:26.964645 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.965179 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.965951 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.966152 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.966242 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.966443 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:26.966543 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:26.966639 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.966682 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.966703 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:36:26.968492 28033 raft_consensus.cc:1261] T 064191579c994edc96f03fd413d22380 P 047ab3185fee4b09820be6c918bfa681 [term 1 FOLLOWER]: Refusing update from remote peer 6784121b23d7441f88f6e0b048105cdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:26.968490 27887 raft_consensus.cc:1261] T 064191579c994edc96f03fd413d22380 P eaefcaa82f5e45039d58b6493a21ba89 [term 1 FOLLOWER]: Refusing update from remote peer 6784121b23d7441f88f6e0b048105cdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:26.968762 28125 consensus_queue.cc:1035] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:26.968900 28109 consensus_queue.cc:1035] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:26.969947 28107 tablet_replica.cc:439] T 064191579c994edc96f03fd413d22380 P 047ab3185fee4b09820be6c918bfa681: TxnStatusTablet state changed. Reason: New leader 6784121b23d7441f88f6e0b048105cdf. Latest consensus state: current_term: 1 leader_uuid: "6784121b23d7441f88f6e0b048105cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } } }
I20260619 20:36:26.970052 28107 tablet_replica.cc:442] T 064191579c994edc96f03fd413d22380 P 047ab3185fee4b09820be6c918bfa681: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:26.970022 28110 tablet_replica.cc:439] T 064191579c994edc96f03fd413d22380 P eaefcaa82f5e45039d58b6493a21ba89: TxnStatusTablet state changed. Reason: New leader 6784121b23d7441f88f6e0b048105cdf. Latest consensus state: current_term: 1 leader_uuid: "6784121b23d7441f88f6e0b048105cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } } }
I20260619 20:36:26.970101 28110 tablet_replica.cc:442] T 064191579c994edc96f03fd413d22380 P eaefcaa82f5e45039d58b6493a21ba89: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:26.971213 28125 tablet_replica.cc:439] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6784121b23d7441f88f6e0b048105cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } } }
I20260619 20:36:26.971302 28125 tablet_replica.cc:442] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:36:26.971411 28125 tablet_replica.cc:439] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6784121b23d7441f88f6e0b048105cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } } }
I20260619 20:36:26.971484 28125 tablet_replica.cc:442] T 064191579c994edc96f03fd413d22380 P 6784121b23d7441f88f6e0b048105cdf: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:36:26.971479 28107 tablet_replica.cc:439] T 064191579c994edc96f03fd413d22380 P 047ab3185fee4b09820be6c918bfa681: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6784121b23d7441f88f6e0b048105cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } } }
I20260619 20:36:26.971554 28107 tablet_replica.cc:442] T 064191579c994edc96f03fd413d22380 P 047ab3185fee4b09820be6c918bfa681: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:26.971737 28110 tablet_replica.cc:439] T 064191579c994edc96f03fd413d22380 P eaefcaa82f5e45039d58b6493a21ba89: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6784121b23d7441f88f6e0b048105cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } } }
I20260619 20:36:26.971812 28110 tablet_replica.cc:442] T 064191579c994edc96f03fd413d22380 P eaefcaa82f5e45039d58b6493a21ba89: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:26.971848 28131 mvcc.cc:205] Tried to move back new op lower bound from 7298668081020145664 to 7298668080971763712. Current Snapshot: MvccSnapshot[applied={T|T < 7298668081020145664}]
I20260619 20:36:26.972571 28132 mvcc.cc:205] Tried to move back new op lower bound from 7298668081020145664 to 7298668080971763712. Current Snapshot: MvccSnapshot[applied={T|T < 7298668081020145664}]
I20260619 20:36:26.977432 28098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.974861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55444 (local address 127.25.62.131:44623)
0619 20:36:26.974904 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.974906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.974914 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:26.975005 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.975006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.975022 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.975116 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:26.975268 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.975804 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.976561 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.976751 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.976845 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.977068 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:26.977162 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:26.977286 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.977334 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.977354 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:36:26.982884 28085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.980140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:38485 (local address 127.25.62.131:46335)
0619 20:36:26.980220 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:26.980231 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:26.980269 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:26.980424 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:26.980424 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:26.980425 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:36:26.980481 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.980482 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.981406 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.981407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.981972 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.981974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.982456 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.982457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.982511 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.982525 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:36:26.982754 (+ 229us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:36:26.982755 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 20:36:26.982800 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 20:36:26.982834 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 20:36:26.982914 28101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.980244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:46335 (local address 127.25.62.129:38485)
0619 20:36:26.980287 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.980289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.980300 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:36:26.980328 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.980329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.980340 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.980418 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:26.980560 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.981306 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.982070 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.982361 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.982441 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.982653 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:26.982690 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:26.982748 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.982806 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.982827 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:36:26.985846 27887 raft_consensus.cc:1261] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89 [term 1 FOLLOWER]: Refusing update from remote peer 047ab3185fee4b09820be6c918bfa681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:26.985942 27750 raft_consensus.cc:1261] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf [term 1 FOLLOWER]: Refusing update from remote peer 047ab3185fee4b09820be6c918bfa681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:26.986120 28107 consensus_queue.cc:1035] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:26.986277 28112 consensus_queue.cc:1035] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:26.995337 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.992369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:44623 (local address 127.25.62.129:58619)
0619 20:36:26.992575 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:26.992586 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:26.992634 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:26.992779 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:26.992779 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:26.992781 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:36:26.992828 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.992829 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.993712 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.993713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.994505 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:26.994507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.994980 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.994981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:26.995024 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.995036 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:36:26.995206 (+ 170us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:36:26.995208 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:36:26.995238 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 20:36:26.995269 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":85,"threads_started":1}
I20260619 20:36:26.995343 28098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:26.992569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:58619 (local address 127.25.62.131:44623)
0619 20:36:26.992610 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:26.992612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:26.992624 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:36:26.992700 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:26.992701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:26.992711 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:26.992780 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:26.992875 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.993618 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.994588 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:26.994900 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:26.994981 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:26.995112 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:26.995147 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:26.995202 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:26.995254 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 20:36:26.995274 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:36:27.004195 27593 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:27.004313 27593 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:27.005151 27593 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9: Sending DeleteTablet for 3 replicas of tablet bc5b7c4d755544208de357007ce04e4a
I20260619 20:36:27.005467 28013 tablet_service.cc:1514] Processing DeleteTablet for tablet bc5b7c4d755544208de357007ce04e4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:26 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:55438
I20260619 20:36:27.005559 27870 tablet_service.cc:1514] Processing DeleteTablet for tablet bc5b7c4d755544208de357007ce04e4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:26 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:38222
I20260619 20:36:27.005748 27730 tablet_service.cc:1514] Processing DeleteTablet for tablet bc5b7c4d755544208de357007ce04e4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:26 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:56306
I20260619 20:36:27.006057 27593 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:36:27.006206 27593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:36:27.007428 27730 tablet_service.cc:1467] Processing CreateTablet for tablet 3d640de65cc847168b58a96234b06160 (DEFAULT_TABLE table=test-table [id=eef66a4e12214d469c92df4661af0c03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:27.007459 27870 tablet_service.cc:1467] Processing CreateTablet for tablet 3d640de65cc847168b58a96234b06160 (DEFAULT_TABLE table=test-table [id=eef66a4e12214d469c92df4661af0c03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:27.007560 27730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d640de65cc847168b58a96234b06160. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:27.007584 27870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d640de65cc847168b58a96234b06160. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:27.008845 28106 tablet_bootstrap.cc:492] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89: Bootstrap starting.
I20260619 20:36:27.008870 28102 tablet_bootstrap.cc:492] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf: Bootstrap starting.
I20260619 20:36:27.009313 28106 tablet_bootstrap.cc:654] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:27.009500 28102 tablet_bootstrap.cc:654] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:27.010042 28013 tablet_service.cc:1467] Processing CreateTablet for tablet 3d640de65cc847168b58a96234b06160 (DEFAULT_TABLE table=test-table [id=eef66a4e12214d469c92df4661af0c03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:27.010090 28106 tablet_bootstrap.cc:492] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89: No bootstrap required, opened a new log
I20260619 20:36:27.010135 28106 ts_tablet_manager.cc:1397] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:27.010147 28013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d640de65cc847168b58a96234b06160. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:27.010273 28106 raft_consensus.cc:348] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:27.010339 28106 raft_consensus.cc:374] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:27.010382 28106 raft_consensus.cc:729] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaefcaa82f5e45039d58b6493a21ba89, State: Initialized, Role: FOLLOWER
I20260619 20:36:27.010428 28106 consensus_queue.cc:260] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:27.010526 28106 ts_tablet_manager.cc:1428] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:27.011349 28149 tablet_replica.cc:330] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681: stopping tablet replica
I20260619 20:36:27.011449 28149 raft_consensus.cc:2229] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:36:27.011451 28103 tablet_bootstrap.cc:492] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681: Bootstrap starting.
I20260619 20:36:27.011557 28149 raft_consensus.cc:2258] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:27.011947 28103 tablet_bootstrap.cc:654] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:27.012009 28149 ts_tablet_manager.cc:1905] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:27.012348 28102 tablet_bootstrap.cc:492] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf: No bootstrap required, opened a new log
I20260619 20:36:27.012399 28102 ts_tablet_manager.cc:1397] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260619 20:36:27.012569 28102 raft_consensus.cc:348] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:27.012642 28102 raft_consensus.cc:374] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:27.012663 28102 raft_consensus.cc:729] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6784121b23d7441f88f6e0b048105cdf, State: Initialized, Role: FOLLOWER
I20260619 20:36:27.012688 28103 tablet_bootstrap.cc:492] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681: No bootstrap required, opened a new log
I20260619 20:36:27.012702 28102 consensus_queue.cc:260] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:27.012732 28103 ts_tablet_manager.cc:1397] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:27.012781 28102 ts_tablet_manager.cc:1428] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:27.012868 28103 raft_consensus.cc:348] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:27.012953 28103 raft_consensus.cc:374] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:27.012977 28103 raft_consensus.cc:729] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047ab3185fee4b09820be6c918bfa681, State: Initialized, Role: FOLLOWER
I20260619 20:36:27.013019 28103 consensus_queue.cc:260] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:27.013126 28151 tablet_replica.cc:330] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf: stopping tablet replica
I20260619 20:36:27.013144 28103 ts_tablet_manager.cc:1428] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:27.013233 28151 raft_consensus.cc:2229] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:27.013288 28151 raft_consensus.cc:2258] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:27.013435 28150 tablet_replica.cc:330] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89: stopping tablet replica
I20260619 20:36:27.013547 28150 raft_consensus.cc:2229] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:27.013610 28150 raft_consensus.cc:2258] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:27.014204 28150 ts_tablet_manager.cc:1905] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:27.015878 28150 ts_tablet_manager.cc:1918] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:36:27.015969 28150 log.cc:1196] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/bc5b7c4d755544208de357007ce04e4a
I20260619 20:36:27.016247 28150 ts_tablet_manager.cc:1939] T bc5b7c4d755544208de357007ce04e4a P eaefcaa82f5e45039d58b6493a21ba89: Deleting consensus metadata
I20260619 20:36:27.016680 27578 catalog_manager.cc:4841] TS eaefcaa82f5e45039d58b6493a21ba89 (127.25.62.130:39075): tablet bc5b7c4d755544208de357007ce04e4a (table test-table [id=0d4733556bcd4463a55488bf31c4f406]) successfully deleted
I20260619 20:36:27.021270 28151 ts_tablet_manager.cc:1905] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:27.022710 28151 ts_tablet_manager.cc:1918] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:36:27.022790 28151 log.cc:1196] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/wal/wals/bc5b7c4d755544208de357007ce04e4a
I20260619 20:36:27.023101 28151 ts_tablet_manager.cc:1939] T bc5b7c4d755544208de357007ce04e4a P 6784121b23d7441f88f6e0b048105cdf: Deleting consensus metadata
I20260619 20:36:27.023540 27578 catalog_manager.cc:4841] TS 6784121b23d7441f88f6e0b048105cdf (127.25.62.129:38485): tablet bc5b7c4d755544208de357007ce04e4a (table test-table [id=0d4733556bcd4463a55488bf31c4f406]) successfully deleted
I20260619 20:36:27.025534 28149 ts_tablet_manager.cc:1918] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:36:27.025631 28149 log.cc:1196] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/bc5b7c4d755544208de357007ce04e4a
I20260619 20:36:27.025857 28149 ts_tablet_manager.cc:1939] T bc5b7c4d755544208de357007ce04e4a P 047ab3185fee4b09820be6c918bfa681: Deleting consensus metadata
I20260619 20:36:27.026427 27580 catalog_manager.cc:4841] TS 047ab3185fee4b09820be6c918bfa681 (127.25.62.131:44623): tablet bc5b7c4d755544208de357007ce04e4a (table test-table [id=0d4733556bcd4463a55488bf31c4f406]) successfully deleted
I20260619 20:36:27.047643 28110 raft_consensus.cc:482] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:36:27.047744 28110 raft_consensus.cc:504] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:27.048065 28110 leader_election.cc:288] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6784121b23d7441f88f6e0b048105cdf (127.25.62.129:38485), 047ab3185fee4b09820be6c918bfa681 (127.25.62.131:44623)
I20260619 20:36:27.052316 28098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:27.048407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:42149 (local address 127.25.62.131:44623)
0619 20:36:27.048487 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:27.048489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:27.048698 (+ 209us) server_negotiation.cc:407] Connection header received
0619 20:36:27.048789 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:27.048790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:27.048799 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:27.048827 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:27.049060 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:27.049821 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:27.051358 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:27.051925 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:27.051971 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:27.051980 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:27.052211 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 20:36:27.052245 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 20:36:27.052388 28101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:27.048428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:59589 (local address 127.25.62.129:38485)
0619 20:36:27.048526 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:27.048528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:27.048579 (+ 51us) server_negotiation.cc:407] Connection header received
0619 20:36:27.048647 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:27.048648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:27.048659 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:27.048725 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:27.049352 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:27.050027 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:27.051426 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:27.051952 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:27.052008 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:27.052018 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:27.052278 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 20:36:27.052314 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 20:36:27.052574 28155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:27.048270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:44623 (local address 127.25.62.130:42149)
0619 20:36:27.048653 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:27.048661 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:27.048719 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:27.048887 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:27.048888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:27.048889 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:27.048963 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:27.048965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:27.049919 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:27.049919 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:27.051251 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:27.051254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:27.052058 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:27.052059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:27.052130 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:27.052134 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:27.052428 (+ 294us) client_negotiation.cc:240] Negotiation successful
0619 20:36:27.052486 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":69,"thread_start_us":48,"threads_started":1}
I20260619 20:36:27.052779 28033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d640de65cc847168b58a96234b06160" candidate_uuid: "eaefcaa82f5e45039d58b6493a21ba89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047ab3185fee4b09820be6c918bfa681" is_pre_election: true
I20260619 20:36:27.052865 28033 raft_consensus.cc:2454] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaefcaa82f5e45039d58b6493a21ba89 in term 0.
I20260619 20:36:27.053011 27824 leader_election.cc:302] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047ab3185fee4b09820be6c918bfa681, eaefcaa82f5e45039d58b6493a21ba89; no voters:
I20260619 20:36:27.053157 28110 raft_consensus.cc:2781] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:36:27.053251 28110 raft_consensus.cc:482] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:36:27.053278 28110 raft_consensus.cc:3037] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:27.054225 28110 raft_consensus.cc:504] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:27.054387 28110 leader_election.cc:288] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [CANDIDATE]: Term 1 election: Requested vote from peers 6784121b23d7441f88f6e0b048105cdf (127.25.62.129:38485), 047ab3185fee4b09820be6c918bfa681 (127.25.62.131:44623)
I20260619 20:36:27.054534 28156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:27.048250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:38485 (local address 127.25.62.130:59589)
0619 20:36:27.048509 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:27.048521 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:27.048581 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:27.049165 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:27.049166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:27.049168 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:27.049248 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:27.049250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:27.050147 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:27.050147 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:27.051305 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:27.051309 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:27.052086 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:27.052087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:27.052180 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:27.052184 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:27.052993 (+ 809us) client_negotiation.cc:240] Negotiation successful
0619 20:36:27.053072 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":72,"threads_started":1}
I20260619 20:36:27.054538 28033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d640de65cc847168b58a96234b06160" candidate_uuid: "eaefcaa82f5e45039d58b6493a21ba89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047ab3185fee4b09820be6c918bfa681"
I20260619 20:36:27.054601 28033 raft_consensus.cc:3037] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:27.054903 27750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d640de65cc847168b58a96234b06160" candidate_uuid: "eaefcaa82f5e45039d58b6493a21ba89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6784121b23d7441f88f6e0b048105cdf" is_pre_election: true
I20260619 20:36:27.055001 27750 raft_consensus.cc:2454] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaefcaa82f5e45039d58b6493a21ba89 in term 0.
I20260619 20:36:27.054905 27749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d640de65cc847168b58a96234b06160" candidate_uuid: "eaefcaa82f5e45039d58b6493a21ba89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6784121b23d7441f88f6e0b048105cdf"
I20260619 20:36:27.055145 27749 raft_consensus.cc:3037] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:27.055367 28033 raft_consensus.cc:2454] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaefcaa82f5e45039d58b6493a21ba89 in term 1.
I20260619 20:36:27.055708 27824 leader_election.cc:302] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047ab3185fee4b09820be6c918bfa681, eaefcaa82f5e45039d58b6493a21ba89; no voters:
I20260619 20:36:27.055817 28110 raft_consensus.cc:2781] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:27.055935 28110 raft_consensus.cc:686] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [term 1 LEADER]: Becoming Leader. State: Replica: eaefcaa82f5e45039d58b6493a21ba89, State: Running, Role: LEADER
I20260619 20:36:27.055990 28110 consensus_queue.cc:237] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:27.056053 27749 raft_consensus.cc:2454] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaefcaa82f5e45039d58b6493a21ba89 in term 1.
I20260619 20:36:27.056646 27593 catalog_manager.cc:5495] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 reported cstate change: term changed from 0 to 1, leader changed from <none> to eaefcaa82f5e45039d58b6493a21ba89 (127.25.62.130). New cstate: current_term: 1 leader_uuid: "eaefcaa82f5e45039d58b6493a21ba89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } health_report { overall_health: UNKNOWN } } }
I20260619 20:36:27.065580 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:27.062320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38226 (local address 127.25.62.130:39075)
0619 20:36:27.062421 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:27.062423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:27.062430 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:36:27.062457 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:27.062457 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:27.062486 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:27.062557 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:27.062713 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:27.063441 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:27.064535 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:27.064827 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:27.064883 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:27.065082 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:27.065202 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:27.065266 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:27.065492 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 20:36:27.065510 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 20:36:27.071332 28101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:27.068128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:55117 (local address 127.25.62.129:38485)
0619 20:36:27.068172 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:27.068173 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:27.068355 (+ 182us) server_negotiation.cc:407] Connection header received
0619 20:36:27.068422 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:27.068423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:27.068435 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:27.068502 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:27.068670 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:27.069395 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:27.070491 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:27.070766 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:27.070836 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:27.071062 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:27.071094 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:27.071143 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:27.071241 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 20:36:27.071261 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:36:27.071399 28161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:27.068055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:38485 (local address 127.25.62.130:55117)
0619 20:36:27.068263 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:27.068275 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:27.068351 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:27.068523 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:27.068524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:27.068525 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:36:27.068594 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:27.068595 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:27.069488 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:27.069489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:27.070384 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:27.070387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:27.070864 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:27.070865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:27.070947 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:27.070963 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:36:27.071170 (+ 207us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:36:27.071172 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:36:27.071238 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 20:36:27.071287 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":77,"threads_started":1}
I20260619 20:36:27.073669 27749 raft_consensus.cc:1261] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf [term 1 FOLLOWER]: Refusing update from remote peer eaefcaa82f5e45039d58b6493a21ba89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:27.073699 28033 raft_consensus.cc:1261] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681 [term 1 FOLLOWER]: Refusing update from remote peer eaefcaa82f5e45039d58b6493a21ba89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:27.074146 28157 consensus_queue.cc:1035] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:27.074339 28110 consensus_queue.cc:1035] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:27.079407 28130 mvcc.cc:205] Tried to move back new op lower bound from 7298668081451560960 to 7298668081381785600. Current Snapshot: MvccSnapshot[applied={T|T < 7298668081451560960}]
I20260619 20:36:27.084060 28145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:27.081002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:39075 (local address 127.25.62.129:44513)
0619 20:36:27.081061 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:27.081072 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:27.081112 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:27.081326 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:27.081327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:27.081328 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:36:27.081369 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:27.081370 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:27.082263 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:27.082264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:27.083096 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:27.083099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:27.083617 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:27.083618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:27.083677 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:27.083685 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:36:27.083882 (+ 197us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:36:27.083885 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:36:27.083944 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 20:36:27.083988 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260619 20:36:27.084064 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:27.081114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:44513 (local address 127.25.62.130:39075)
0619 20:36:27.081170 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:27.081172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:27.081181 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:36:27.081204 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:27.081205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:27.081216 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:27.081277 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:27.081450 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:27.082137 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:27.083205 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:27.083497 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:27.083577 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:27.083767 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:27.083801 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:27.083860 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:27.083968 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 20:36:27.083990 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260619 20:36:27.084401 28132 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 20:36:28.607904 27592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:28.608011 27592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:28.608863 27592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9: Sending DeleteTablet for 3 replicas of tablet 3d640de65cc847168b58a96234b06160
I20260619 20:36:28.609148 28013 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d640de65cc847168b58a96234b06160 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:28 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:55438
I20260619 20:36:28.609155 27870 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d640de65cc847168b58a96234b06160 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:28 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:38222
I20260619 20:36:28.609422 27730 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d640de65cc847168b58a96234b06160 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:28 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:56306
I20260619 20:36:28.609601 28192 tablet_replica.cc:330] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf: stopping tablet replica
I20260619 20:36:28.609612 28191 tablet_replica.cc:330] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681: stopping tablet replica
I20260619 20:36:28.609681 28192 raft_consensus.cc:2229] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:28.609683 28191 raft_consensus.cc:2229] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:28.609728 28192 raft_consensus.cc:2258] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:28.609730 28191 raft_consensus.cc:2258] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:28.609853 28190 tablet_replica.cc:330] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89: stopping tablet replica
I20260619 20:36:28.609920 28190 raft_consensus.cc:2229] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:36:28.610008 28192 ts_tablet_manager.cc:1905] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:28.610016 28190 raft_consensus.cc:2258] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:28.610219 28191 ts_tablet_manager.cc:1905] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:28.610368 28190 ts_tablet_manager.cc:1905] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:28.611860 28192 ts_tablet_manager.cc:1918] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 20:36:28.611949 28192 log.cc:1196] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/wal/wals/3d640de65cc847168b58a96234b06160
I20260619 20:36:28.612182 28192 ts_tablet_manager.cc:1939] T 3d640de65cc847168b58a96234b06160 P 6784121b23d7441f88f6e0b048105cdf: Deleting consensus metadata
I20260619 20:36:28.612257 28191 ts_tablet_manager.cc:1918] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 20:36:28.612313 28191 log.cc:1196] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/3d640de65cc847168b58a96234b06160
I20260619 20:36:28.612509 28190 ts_tablet_manager.cc:1918] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 20:36:28.612576 28190 log.cc:1196] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/3d640de65cc847168b58a96234b06160
I20260619 20:36:28.612603 28191 ts_tablet_manager.cc:1939] T 3d640de65cc847168b58a96234b06160 P 047ab3185fee4b09820be6c918bfa681: Deleting consensus metadata
I20260619 20:36:28.612648 27578 catalog_manager.cc:4841] TS 6784121b23d7441f88f6e0b048105cdf (127.25.62.129:38485): tablet 3d640de65cc847168b58a96234b06160 (table test-table [id=eef66a4e12214d469c92df4661af0c03]) successfully deleted
I20260619 20:36:28.612815 28190 ts_tablet_manager.cc:1939] T 3d640de65cc847168b58a96234b06160 P eaefcaa82f5e45039d58b6493a21ba89: Deleting consensus metadata
I20260619 20:36:28.612998 27580 catalog_manager.cc:4841] TS 047ab3185fee4b09820be6c918bfa681 (127.25.62.131:44623): tablet 3d640de65cc847168b58a96234b06160 (table test-table [id=eef66a4e12214d469c92df4661af0c03]) successfully deleted
I20260619 20:36:28.613148 27578 catalog_manager.cc:4841] TS eaefcaa82f5e45039d58b6493a21ba89 (127.25.62.130:39075): tablet 3d640de65cc847168b58a96234b06160 (table test-table [id=eef66a4e12214d469c92df4661af0c03]) successfully deleted
Jun 19 20:36:28 dist-test-slave-0t40 krb5kdc[27548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901388, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 20:36:28 dist-test-slave-0t40 krb5kdc[27548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901388, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:28.638108 28200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:28.626936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60768 (local address 127.25.62.190:35573)
0619 20:36:28.627086 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:28.627088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:28.627096 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:28.627123 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:28.627124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:28.627137 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:28.627194 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:28.627688 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:28.628194 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:28.628866 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:28.629101 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:28.629153 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:28.635850 (+ 6697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:28.635851 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:28.635851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:28.635857 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:28.637195 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:28.637488 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:28.637489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:28.637489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:28.637538 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:28.637685 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:28.637685 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:28.637686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:28.637781 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:28.637852 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:28.637936 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 20:36:28.638005 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260619 20:36:28.639559 27592 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:36:28.639705 27592 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:36:28.640901 27730 tablet_service.cc:1467] Processing CreateTablet for tablet 12abc1903d5f4a178ef0693c05ddc599 (DEFAULT_TABLE table=test-table [id=7b4f2c4d124145a6af8bddc8a28e10ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:28.640937 27870 tablet_service.cc:1467] Processing CreateTablet for tablet 12abc1903d5f4a178ef0693c05ddc599 (DEFAULT_TABLE table=test-table [id=7b4f2c4d124145a6af8bddc8a28e10ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:28.640949 28013 tablet_service.cc:1467] Processing CreateTablet for tablet 12abc1903d5f4a178ef0693c05ddc599 (DEFAULT_TABLE table=test-table [id=7b4f2c4d124145a6af8bddc8a28e10ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:28.641072 27730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12abc1903d5f4a178ef0693c05ddc599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:28.641095 28013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12abc1903d5f4a178ef0693c05ddc599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:28.641115 27870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12abc1903d5f4a178ef0693c05ddc599. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:28.642416 28202 tablet_bootstrap.cc:492] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf: Bootstrap starting.
I20260619 20:36:28.642702 28203 tablet_bootstrap.cc:492] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89: Bootstrap starting.
I20260619 20:36:28.642841 28201 tablet_bootstrap.cc:492] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681: Bootstrap starting.
I20260619 20:36:28.643093 28203 tablet_bootstrap.cc:654] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:28.643214 28201 tablet_bootstrap.cc:654] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:28.643489 28202 tablet_bootstrap.cc:654] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:28.643672 28203 tablet_bootstrap.cc:492] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89: No bootstrap required, opened a new log
I20260619 20:36:28.643714 28203 ts_tablet_manager.cc:1397] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:28.643846 28203 raft_consensus.cc:348] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:28.643929 28203 raft_consensus.cc:374] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:28.643952 28203 raft_consensus.cc:729] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaefcaa82f5e45039d58b6493a21ba89, State: Initialized, Role: FOLLOWER
I20260619 20:36:28.644014 28203 consensus_queue.cc:260] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:28.644115 28203 ts_tablet_manager.cc:1428] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:28.644564 28201 tablet_bootstrap.cc:492] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681: No bootstrap required, opened a new log
I20260619 20:36:28.644627 28201 ts_tablet_manager.cc:1397] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:36:28.644735 28201 raft_consensus.cc:348] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:28.644811 28201 raft_consensus.cc:374] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:28.644832 28201 raft_consensus.cc:729] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047ab3185fee4b09820be6c918bfa681, State: Initialized, Role: FOLLOWER
I20260619 20:36:28.644907 28202 tablet_bootstrap.cc:492] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf: No bootstrap required, opened a new log
I20260619 20:36:28.644956 28202 ts_tablet_manager.cc:1397] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 20:36:28.645107 28202 raft_consensus.cc:348] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:28.645171 28202 raft_consensus.cc:374] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:28.645185 28202 raft_consensus.cc:729] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6784121b23d7441f88f6e0b048105cdf, State: Initialized, Role: FOLLOWER
I20260619 20:36:28.645236 28202 consensus_queue.cc:260] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:28.645344 28202 ts_tablet_manager.cc:1428] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:28.645423 28201 consensus_queue.cc:260] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:28.645612 28201 ts_tablet_manager.cc:1428] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:28.680701 28176 raft_consensus.cc:482] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:36:28.680799 28176 raft_consensus.cc:504] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:28.681005 28176 leader_election.cc:288] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eaefcaa82f5e45039d58b6493a21ba89 (127.25.62.130:39075), 047ab3185fee4b09820be6c918bfa681 (127.25.62.131:44623)
I20260619 20:36:28.681262 27887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12abc1903d5f4a178ef0693c05ddc599" candidate_uuid: "6784121b23d7441f88f6e0b048105cdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaefcaa82f5e45039d58b6493a21ba89" is_pre_election: true
I20260619 20:36:28.681262 28033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12abc1903d5f4a178ef0693c05ddc599" candidate_uuid: "6784121b23d7441f88f6e0b048105cdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047ab3185fee4b09820be6c918bfa681" is_pre_election: true
I20260619 20:36:28.681371 28033 raft_consensus.cc:2454] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6784121b23d7441f88f6e0b048105cdf in term 0.
I20260619 20:36:28.681372 27887 raft_consensus.cc:2454] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6784121b23d7441f88f6e0b048105cdf in term 0.
I20260619 20:36:28.681556 27682 leader_election.cc:302] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6784121b23d7441f88f6e0b048105cdf, eaefcaa82f5e45039d58b6493a21ba89; no voters:
I20260619 20:36:28.681658 28176 raft_consensus.cc:2781] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:36:28.681702 28176 raft_consensus.cc:482] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:36:28.681722 28176 raft_consensus.cc:3037] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:28.682399 28176 raft_consensus.cc:504] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:28.682567 28176 leader_election.cc:288] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [CANDIDATE]: Term 1 election: Requested vote from peers eaefcaa82f5e45039d58b6493a21ba89 (127.25.62.130:39075), 047ab3185fee4b09820be6c918bfa681 (127.25.62.131:44623)
I20260619 20:36:28.682705 27887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12abc1903d5f4a178ef0693c05ddc599" candidate_uuid: "6784121b23d7441f88f6e0b048105cdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaefcaa82f5e45039d58b6493a21ba89"
I20260619 20:36:28.682783 27887 raft_consensus.cc:3037] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:28.682762 28033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12abc1903d5f4a178ef0693c05ddc599" candidate_uuid: "6784121b23d7441f88f6e0b048105cdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047ab3185fee4b09820be6c918bfa681"
I20260619 20:36:28.682866 28033 raft_consensus.cc:3037] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:28.683424 27887 raft_consensus.cc:2454] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6784121b23d7441f88f6e0b048105cdf in term 1.
I20260619 20:36:28.683509 28033 raft_consensus.cc:2454] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6784121b23d7441f88f6e0b048105cdf in term 1.
I20260619 20:36:28.683602 27682 leader_election.cc:302] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6784121b23d7441f88f6e0b048105cdf, eaefcaa82f5e45039d58b6493a21ba89; no voters:
I20260619 20:36:28.683704 28176 raft_consensus.cc:2781] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:28.683766 28176 raft_consensus.cc:686] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [term 1 LEADER]: Becoming Leader. State: Replica: 6784121b23d7441f88f6e0b048105cdf, State: Running, Role: LEADER
I20260619 20:36:28.683837 28176 consensus_queue.cc:237] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:28.684499 27593 catalog_manager.cc:5495] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf reported cstate change: term changed from 0 to 1, leader changed from <none> to 6784121b23d7441f88f6e0b048105cdf (127.25.62.129). New cstate: current_term: 1 leader_uuid: "6784121b23d7441f88f6e0b048105cdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } health_report { overall_health: UNKNOWN } } }
I20260619 20:36:28.691715 28210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:28.689639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56324 (local address 127.25.62.129:38485)
0619 20:36:28.689790 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:28.689792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:28.689799 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:36:28.689817 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:28.689817 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:28.689826 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:28.689884 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:28.690021 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:28.690490 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:28.691183 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:28.691369 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:28.691419 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:28.691536 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:28.691562 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:28.691600 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:28.691642 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 20:36:28.691659 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260619 20:36:28.695663 28210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:28.693689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:58897 (local address 127.25.62.129:38485)
0619 20:36:28.693738 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:28.693740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:28.693748 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:28.693790 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:28.693791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:28.693797 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:28.693843 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:28.693928 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:28.694353 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:28.695110 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:28.695281 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:28.695333 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:28.695459 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:28.695481 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:28.695522 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:28.695579 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 20:36:28.695594 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 20:36:28.695746 28213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:28.693578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:38485 (local address 127.25.62.129:58897)
0619 20:36:28.693692 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:28.693701 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:28.693750 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:28.693849 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:28.693850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:28.693852 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:36:28.693884 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:28.693885 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:28.694445 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:28.694446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:28.695029 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:28.695031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:28.695347 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:28.695348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:28.695397 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:28.695401 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:36:28.695529 (+ 128us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:36:28.695531 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:36:28.695576 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 20:36:28.695613 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":72,"threads_started":1}
I20260619 20:36:28.697494 28033 raft_consensus.cc:1261] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681 [term 1 FOLLOWER]: Refusing update from remote peer 6784121b23d7441f88f6e0b048105cdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:28.697592 27887 raft_consensus.cc:1261] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89 [term 1 FOLLOWER]: Refusing update from remote peer 6784121b23d7441f88f6e0b048105cdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:28.697723 28171 consensus_queue.cc:1035] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:28.697819 28176 consensus_queue.cc:1035] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:28.707975 27592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:28.708074 27592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:28.708652 27592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9: Sending DeleteTablet for 3 replicas of tablet 12abc1903d5f4a178ef0693c05ddc599
I20260619 20:36:28.708869 28013 tablet_service.cc:1514] Processing DeleteTablet for tablet 12abc1903d5f4a178ef0693c05ddc599 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:28 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:55438
I20260619 20:36:28.708907 27870 tablet_service.cc:1514] Processing DeleteTablet for tablet 12abc1903d5f4a178ef0693c05ddc599 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:28 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:38222
I20260619 20:36:28.708971 28191 tablet_replica.cc:330] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681: stopping tablet replica
I20260619 20:36:28.709066 28191 raft_consensus.cc:2229] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:28.709092 28190 tablet_replica.cc:330] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89: stopping tablet replica
I20260619 20:36:28.709116 28191 raft_consensus.cc:2258] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:28.709152 28190 raft_consensus.cc:2229] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:28.709192 28190 raft_consensus.cc:2258] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:28.709254 27730 tablet_service.cc:1514] Processing DeleteTablet for tablet 12abc1903d5f4a178ef0693c05ddc599 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:28 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:56306
I20260619 20:36:28.709335 28192 tablet_replica.cc:330] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf: stopping tablet replica
I20260619 20:36:28.709388 28192 raft_consensus.cc:2229] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:36:28.709476 28192 raft_consensus.cc:2258] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:28.709570 28191 ts_tablet_manager.cc:1905] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:28.709571 27592 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:36:28.709700 27592 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:36:28.709990 28190 ts_tablet_manager.cc:1905] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:28.711378 28192 ts_tablet_manager.cc:1905] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:28.711601 28190 ts_tablet_manager.cc:1918] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:36:28.711651 28190 log.cc:1196] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/12abc1903d5f4a178ef0693c05ddc599
I20260619 20:36:28.711865 28190 ts_tablet_manager.cc:1939] T 12abc1903d5f4a178ef0693c05ddc599 P eaefcaa82f5e45039d58b6493a21ba89: Deleting consensus metadata
I20260619 20:36:28.712296 28013 tablet_service.cc:1467] Processing CreateTablet for tablet 736630f08c5e4e4989550d195a37544b (DEFAULT_TABLE table=test-table [id=a6b4e2078fe649529bc24e952d5a59cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:28.712522 28191 ts_tablet_manager.cc:1918] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:36:28.712567 28191 log.cc:1196] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/12abc1903d5f4a178ef0693c05ddc599
I20260619 20:36:28.712739 28191 ts_tablet_manager.cc:1939] T 12abc1903d5f4a178ef0693c05ddc599 P 047ab3185fee4b09820be6c918bfa681: Deleting consensus metadata
I20260619 20:36:28.712859 28013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 736630f08c5e4e4989550d195a37544b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:28.712929 28192 ts_tablet_manager.cc:1918] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:36:28.712981 28192 log.cc:1196] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-0/wal/wals/12abc1903d5f4a178ef0693c05ddc599
I20260619 20:36:28.713217 28192 ts_tablet_manager.cc:1939] T 12abc1903d5f4a178ef0693c05ddc599 P 6784121b23d7441f88f6e0b048105cdf: Deleting consensus metadata
I20260619 20:36:28.713292 27578 catalog_manager.cc:4841] TS eaefcaa82f5e45039d58b6493a21ba89 (127.25.62.130:39075): tablet 12abc1903d5f4a178ef0693c05ddc599 (table test-table [id=7b4f2c4d124145a6af8bddc8a28e10ab]) successfully deleted
I20260619 20:36:28.713435 27730 tablet_service.cc:1467] Processing CreateTablet for tablet 736630f08c5e4e4989550d195a37544b (DEFAULT_TABLE table=test-table [id=a6b4e2078fe649529bc24e952d5a59cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:28.713527 27730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 736630f08c5e4e4989550d195a37544b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:28.714001 27870 tablet_service.cc:1467] Processing CreateTablet for tablet 736630f08c5e4e4989550d195a37544b (DEFAULT_TABLE table=test-table [id=a6b4e2078fe649529bc24e952d5a59cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:28.714104 27870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 736630f08c5e4e4989550d195a37544b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:28.714653 28201 tablet_bootstrap.cc:492] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681: Bootstrap starting.
I20260619 20:36:28.714953 28201 tablet_bootstrap.cc:654] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:28.714975 28203 tablet_bootstrap.cc:492] T 736630f08c5e4e4989550d195a37544b P eaefcaa82f5e45039d58b6493a21ba89: Bootstrap starting.
I20260619 20:36:28.715173 28202 tablet_bootstrap.cc:492] T 736630f08c5e4e4989550d195a37544b P 6784121b23d7441f88f6e0b048105cdf: Bootstrap starting.
I20260619 20:36:28.715240 27580 catalog_manager.cc:4841] TS 047ab3185fee4b09820be6c918bfa681 (127.25.62.131:44623): tablet 12abc1903d5f4a178ef0693c05ddc599 (table test-table [id=7b4f2c4d124145a6af8bddc8a28e10ab]) successfully deleted
I20260619 20:36:28.715308 28203 tablet_bootstrap.cc:654] T 736630f08c5e4e4989550d195a37544b P eaefcaa82f5e45039d58b6493a21ba89: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:28.715456 28202 tablet_bootstrap.cc:654] T 736630f08c5e4e4989550d195a37544b P 6784121b23d7441f88f6e0b048105cdf: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:28.715817 28203 tablet_bootstrap.cc:492] T 736630f08c5e4e4989550d195a37544b P eaefcaa82f5e45039d58b6493a21ba89: No bootstrap required, opened a new log
I20260619 20:36:28.715890 28203 ts_tablet_manager.cc:1397] T 736630f08c5e4e4989550d195a37544b P eaefcaa82f5e45039d58b6493a21ba89: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:28.715942 27578 catalog_manager.cc:4841] TS 6784121b23d7441f88f6e0b048105cdf (127.25.62.129:38485): tablet 12abc1903d5f4a178ef0693c05ddc599 (table test-table [id=7b4f2c4d124145a6af8bddc8a28e10ab]) successfully deleted
I20260619 20:36:28.716007 28203 raft_consensus.cc:348] T 736630f08c5e4e4989550d195a37544b P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:28.716079 28203 raft_consensus.cc:374] T 736630f08c5e4e4989550d195a37544b P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:28.716104 28203 raft_consensus.cc:729] T 736630f08c5e4e4989550d195a37544b P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaefcaa82f5e45039d58b6493a21ba89, State: Initialized, Role: FOLLOWER
I20260619 20:36:28.716156 28203 consensus_queue.cc:260] T 736630f08c5e4e4989550d195a37544b P eaefcaa82f5e45039d58b6493a21ba89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:28.716181 28201 tablet_bootstrap.cc:492] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681: No bootstrap required, opened a new log
I20260619 20:36:28.716220 28201 ts_tablet_manager.cc:1397] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:36:28.716251 28203 ts_tablet_manager.cc:1428] T 736630f08c5e4e4989550d195a37544b P eaefcaa82f5e45039d58b6493a21ba89: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:28.716320 28201 raft_consensus.cc:348] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:28.716378 28201 raft_consensus.cc:374] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:28.716413 28201 raft_consensus.cc:729] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047ab3185fee4b09820be6c918bfa681, State: Initialized, Role: FOLLOWER
I20260619 20:36:28.716459 28201 consensus_queue.cc:260] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:28.716573 28201 ts_tablet_manager.cc:1428] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:28.716845 28202 tablet_bootstrap.cc:492] T 736630f08c5e4e4989550d195a37544b P 6784121b23d7441f88f6e0b048105cdf: No bootstrap required, opened a new log
I20260619 20:36:28.716890 28202 ts_tablet_manager.cc:1397] T 736630f08c5e4e4989550d195a37544b P 6784121b23d7441f88f6e0b048105cdf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:36:28.717003 28202 raft_consensus.cc:348] T 736630f08c5e4e4989550d195a37544b P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:28.717088 28202 raft_consensus.cc:374] T 736630f08c5e4e4989550d195a37544b P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:28.717113 28202 raft_consensus.cc:729] T 736630f08c5e4e4989550d195a37544b P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6784121b23d7441f88f6e0b048105cdf, State: Initialized, Role: FOLLOWER
I20260619 20:36:28.717162 28202 consensus_queue.cc:260] T 736630f08c5e4e4989550d195a37544b P 6784121b23d7441f88f6e0b048105cdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:28.717253 28202 ts_tablet_manager.cc:1428] T 736630f08c5e4e4989550d195a37544b P 6784121b23d7441f88f6e0b048105cdf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:29.007962 28207 raft_consensus.cc:482] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:36:29.008057 28207 raft_consensus.cc:504] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:29.008188 28207 leader_election.cc:288] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6784121b23d7441f88f6e0b048105cdf (127.25.62.129:38485), eaefcaa82f5e45039d58b6493a21ba89 (127.25.62.130:39075)
I20260619 20:36:29.008425 27749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "736630f08c5e4e4989550d195a37544b" candidate_uuid: "047ab3185fee4b09820be6c918bfa681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6784121b23d7441f88f6e0b048105cdf" is_pre_election: true
I20260619 20:36:29.008450 27887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "736630f08c5e4e4989550d195a37544b" candidate_uuid: "047ab3185fee4b09820be6c918bfa681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaefcaa82f5e45039d58b6493a21ba89" is_pre_election: true
I20260619 20:36:29.008525 27749 raft_consensus.cc:2454] T 736630f08c5e4e4989550d195a37544b P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 047ab3185fee4b09820be6c918bfa681 in term 0.
I20260619 20:36:29.008562 27887 raft_consensus.cc:2454] T 736630f08c5e4e4989550d195a37544b P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 047ab3185fee4b09820be6c918bfa681 in term 0.
I20260619 20:36:29.008708 27964 leader_election.cc:302] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047ab3185fee4b09820be6c918bfa681, eaefcaa82f5e45039d58b6493a21ba89; no voters:
I20260619 20:36:29.008817 28207 raft_consensus.cc:2781] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:36:29.008882 28207 raft_consensus.cc:482] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:36:29.008935 28207 raft_consensus.cc:3037] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:29.009651 28207 raft_consensus.cc:504] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:29.009779 28207 leader_election.cc:288] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [CANDIDATE]: Term 1 election: Requested vote from peers 6784121b23d7441f88f6e0b048105cdf (127.25.62.129:38485), eaefcaa82f5e45039d58b6493a21ba89 (127.25.62.130:39075)
I20260619 20:36:29.009958 27749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "736630f08c5e4e4989550d195a37544b" candidate_uuid: "047ab3185fee4b09820be6c918bfa681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6784121b23d7441f88f6e0b048105cdf"
I20260619 20:36:29.010035 27749 raft_consensus.cc:3037] T 736630f08c5e4e4989550d195a37544b P 6784121b23d7441f88f6e0b048105cdf [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:29.009994 27887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "736630f08c5e4e4989550d195a37544b" candidate_uuid: "047ab3185fee4b09820be6c918bfa681" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaefcaa82f5e45039d58b6493a21ba89"
I20260619 20:36:29.010076 27887 raft_consensus.cc:3037] T 736630f08c5e4e4989550d195a37544b P eaefcaa82f5e45039d58b6493a21ba89 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:29.010671 27749 raft_consensus.cc:2454] T 736630f08c5e4e4989550d195a37544b P 6784121b23d7441f88f6e0b048105cdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 047ab3185fee4b09820be6c918bfa681 in term 1.
I20260619 20:36:29.010835 27887 raft_consensus.cc:2454] T 736630f08c5e4e4989550d195a37544b P eaefcaa82f5e45039d58b6493a21ba89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 047ab3185fee4b09820be6c918bfa681 in term 1.
I20260619 20:36:29.010864 27964 leader_election.cc:302] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047ab3185fee4b09820be6c918bfa681, 6784121b23d7441f88f6e0b048105cdf; no voters:
I20260619 20:36:29.010967 28207 raft_consensus.cc:2781] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:29.011097 28207 raft_consensus.cc:686] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [term 1 LEADER]: Becoming Leader. State: Replica: 047ab3185fee4b09820be6c918bfa681, State: Running, Role: LEADER
I20260619 20:36:29.011169 28207 consensus_queue.cc:237] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } }
I20260619 20:36:29.011756 27592 catalog_manager.cc:5495] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 reported cstate change: term changed from 0 to 1, leader changed from <none> to 047ab3185fee4b09820be6c918bfa681 (127.25.62.131). New cstate: current_term: 1 leader_uuid: "047ab3185fee4b09820be6c918bfa681" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "047ab3185fee4b09820be6c918bfa681" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44623 } health_report { overall_health: HEALTHY } } }
I20260619 20:36:29.070843 28222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:29.068637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55454 (local address 127.25.62.131:44623)
0619 20:36:29.068805 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:29.068807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:29.068812 (+ 5us) server_negotiation.cc:407] Connection header received
0619 20:36:29.068829 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:29.068830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:29.068839 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:29.068888 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:29.069013 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:29.069558 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:29.070286 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:29.070496 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:29.070556 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:29.070667 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:29.070694 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:29.070732 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:29.070763 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 20:36:29.070780 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":117,"threads_started":1}
I20260619 20:36:29.072865 27877 raft_consensus.cc:1261] T 736630f08c5e4e4989550d195a37544b P eaefcaa82f5e45039d58b6493a21ba89 [term 1 FOLLOWER]: Refusing update from remote peer 047ab3185fee4b09820be6c918bfa681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:29.072872 27749 raft_consensus.cc:1261] T 736630f08c5e4e4989550d195a37544b P 6784121b23d7441f88f6e0b048105cdf [term 1 FOLLOWER]: Refusing update from remote peer 047ab3185fee4b09820be6c918bfa681: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:29.073091 28220 consensus_queue.cc:1035] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaefcaa82f5e45039d58b6493a21ba89" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:29.073539 28220 consensus_queue.cc:1035] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6784121b23d7441f88f6e0b048105cdf" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 20:36:29.077353 28172 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 20:36:30.594300 27593 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:30.594409 27593 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:30.595175 27593 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 71d4233cac234f918e9faa7384a1fbb9: Sending DeleteTablet for 3 replicas of tablet 736630f08c5e4e4989550d195a37544b
I20260619 20:36:30.595404 27730 tablet_service.cc:1514] Processing DeleteTablet for tablet 736630f08c5e4e4989550d195a37544b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:30 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:56306
I20260619 20:36:30.595432 27870 tablet_service.cc:1514] Processing DeleteTablet for tablet 736630f08c5e4e4989550d195a37544b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:30 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:38222
I20260619 20:36:30.595450 28013 tablet_service.cc:1514] Processing DeleteTablet for tablet 736630f08c5e4e4989550d195a37544b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:30 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:55438
I20260619 20:36:30.595595 28247 tablet_replica.cc:330] T 736630f08c5e4e4989550d195a37544b P 6784121b23d7441f88f6e0b048105cdf: stopping tablet replica
I20260619 20:36:30.595608 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 27665
I20260619 20:36:30.595659 28247 raft_consensus.cc:2229] T 736630f08c5e4e4989550d195a37544b P 6784121b23d7441f88f6e0b048105cdf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:30.601164 28249 tablet_replica.cc:330] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681: stopping tablet replica
I20260619 20:36:30.601223 28249 raft_consensus.cc:2229] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:36:30.601287 28249 raft_consensus.cc:2258] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:30.601346 28248 tablet_replica.cc:330] T 736630f08c5e4e4989550d195a37544b P eaefcaa82f5e45039d58b6493a21ba89: stopping tablet replica
I20260619 20:36:30.601421 28248 raft_consensus.cc:2229] T 736630f08c5e4e4989550d195a37544b P eaefcaa82f5e45039d58b6493a21ba89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:30.601467 28248 raft_consensus.cc:2258] T 736630f08c5e4e4989550d195a37544b P eaefcaa82f5e45039d58b6493a21ba89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:30.601629 28249 ts_tablet_manager.cc:1905] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:30.601780 28248 ts_tablet_manager.cc:1905] T 736630f08c5e4e4989550d195a37544b P eaefcaa82f5e45039d58b6493a21ba89: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 20:36:30.602180 27578 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.62.129:38485 (error 108)
I20260619 20:36:30.602372 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 27805
I20260619 20:36:30.603564 28249 ts_tablet_manager.cc:1918] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 20:36:30.603636 28249 log.cc:1196] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/736630f08c5e4e4989550d195a37544b
I20260619 20:36:30.603883 28249 ts_tablet_manager.cc:1939] T 736630f08c5e4e4989550d195a37544b P 047ab3185fee4b09820be6c918bfa681: Deleting consensus metadata
I20260619 20:36:30.604031 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:30.603809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:38485 (local address 127.0.0.1:56338)
0619 20:36:30.603928 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:30.603982 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.129:38485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":77,"threads_started":1}
I20260619 20:36:30.604207 27580 catalog_manager.cc:4841] TS 047ab3185fee4b09820be6c918bfa681 (127.25.62.131:44623): tablet 736630f08c5e4e4989550d195a37544b (table test-table [id=a6b4e2078fe649529bc24e952d5a59cd]) successfully deleted
W20260619 20:36:30.604332 27578 catalog_manager.cc:4584] TS 6784121b23d7441f88f6e0b048105cdf (127.25.62.129:38485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 736630f08c5e4e4989550d195a37544b: Network error: Client connection negotiation failed: client connection to 127.25.62.129:38485: connect: Connection refused (error 111)
I20260619 20:36:30.608259 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:30.608152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:39075 (local address 127.0.0.1:38238)
0619 20:36:30.608200 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:30.608228 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.130:39075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 20:36:30.608309 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 27947
I20260619 20:36:30.614604 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 27561
2026-06-19T20:36:30Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5814 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:30 dist-test-slave-0t40 krb5kdc[28254](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:30 dist-test-slave-0t40 krb5kdc[28254](info): set up 2 sockets
Jun 19 20:36:30 dist-test-slave-0t40 krb5kdc[28254](info): commencing operation
krb5kdc: starting...
W20260619 20:36:31.657130 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:36:31 dist-test-slave-0t40 krb5kdc[28254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:36:31.770308 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:40985
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:40985
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:31.848131 28267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:31.848361 28267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:31.848397 28267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:31.849545 28267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:31.849612 28267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:31.849637 28267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:31.849665 28267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:31.849691 28267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:31.851126 28267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:40985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:40985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:31.851395 28267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:31.851612 28267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:31.856621 28275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:31.856694 28267 server_base.cc:1034] running on GCE node
W20260619 20:36:31.856644 28273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:31.856666 28272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:31.857129 28267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:31.857378 28267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:31.858512 28267 hybrid_clock.cc:648] HybridClock initialized: now 1781901391858501 us; error 28 us; skew 500 ppm
Jun 19 20:36:31 dist-test-slave-0t40 krb5kdc[28254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901391, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:31.860793 28267 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:36:31.861230 28267 webserver.cc:460] Webserver started at http://127.25.62.190:43365/ using document root <none> and password file <none>
I20260619 20:36:31.861464 28267 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:31.861528 28267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:31.861655 28267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:31.862646 28267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "7793fdbccb3c4a7a8fb5e76e704d4880"
format_stamp: "Formatted at 2026-06-19 20:36:31 on dist-test-slave-0t40"
server_key: "6a5f3b0f22aaafb51bd1190f6b4e7a76"
server_key_iv: "d3d652467a6552c498511a95234f0f2b"
server_key_version: "encryptionkey@0"
I20260619 20:36:31.862989 28267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "7793fdbccb3c4a7a8fb5e76e704d4880"
format_stamp: "Formatted at 2026-06-19 20:36:31 on dist-test-slave-0t40"
server_key: "6a5f3b0f22aaafb51bd1190f6b4e7a76"
server_key_iv: "d3d652467a6552c498511a95234f0f2b"
server_key_version: "encryptionkey@0"
I20260619 20:36:31.864228 28267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:36:31.864899 28282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:31.865087 28267 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:31.865172 28267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "7793fdbccb3c4a7a8fb5e76e704d4880"
format_stamp: "Formatted at 2026-06-19 20:36:31 on dist-test-slave-0t40"
server_key: "6a5f3b0f22aaafb51bd1190f6b4e7a76"
server_key_iv: "d3d652467a6552c498511a95234f0f2b"
server_key_version: "encryptionkey@0"
I20260619 20:36:31.865249 28267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:31.887449 28267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:31.889734 28267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:31.889891 28267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:31.893543 28267 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:40985
I20260619 20:36:31.893579 28334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:40985 every 8 connection(s)
I20260619 20:36:31.893927 28267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:31.894474 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 28267
I20260619 20:36:31.894546 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:36:31.894742 25850 external_mini_cluster.cc:1428] Setting key 407511250880859f31fb33254164505c
I20260619 20:36:31.894856 28335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:31.898202 28335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880: Bootstrap starting.
I20260619 20:36:31.898921 28335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:31.899219 28335 log.cc:824] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:31.899926 28335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880: No bootstrap required, opened a new log
Jun 19 20:36:31 dist-test-slave-0t40 krb5kdc[28254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:31.902403 28335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7793fdbccb3c4a7a8fb5e76e704d4880" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 40985 } }
I20260619 20:36:31.902556 28335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:31.902604 28335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7793fdbccb3c4a7a8fb5e76e704d4880, State: Initialized, Role: FOLLOWER
I20260619 20:36:31.902730 28335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7793fdbccb3c4a7a8fb5e76e704d4880" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 40985 } }
I20260619 20:36:31.902830 28335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:31.902865 28335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:31.902920 28335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:31.903622 28335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7793fdbccb3c4a7a8fb5e76e704d4880" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 40985 } }
I20260619 20:36:31.903735 28335 leader_election.cc:302] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7793fdbccb3c4a7a8fb5e76e704d4880; no voters:
I20260619 20:36:31.903895 28335 leader_election.cc:288] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:31.903977 28340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:31.904163 28340 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880 [term 1 LEADER]: Becoming Leader. State: Replica: 7793fdbccb3c4a7a8fb5e76e704d4880, State: Running, Role: LEADER
I20260619 20:36:31.904233 28335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:31.904256 28340 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7793fdbccb3c4a7a8fb5e76e704d4880" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 40985 } }
I20260619 20:36:31.904553 28340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7793fdbccb3c4a7a8fb5e76e704d4880. Latest consensus state: current_term: 1 leader_uuid: "7793fdbccb3c4a7a8fb5e76e704d4880" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7793fdbccb3c4a7a8fb5e76e704d4880" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 40985 } } }
I20260619 20:36:31.904634 28340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:31.904564 28341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7793fdbccb3c4a7a8fb5e76e704d4880" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7793fdbccb3c4a7a8fb5e76e704d4880" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 40985 } } }
I20260619 20:36:31.904892 28341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880 [sys.catalog]: This master's current role is: LEADER
W20260619 20:36:31.906180 28356 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:36:31.906272 28356 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:36:31.906339 28345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:31.906361 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:31.895219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38550 (local address 127.25.62.190:40985)
0619 20:36:31.895413 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:31.895423 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:31.895458 (+ 35us) server_negotiation.cc:407] Connection header received
0619 20:36:31.896033 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:31.896042 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:31.896104 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:31.896256 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:31.896899 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:31.898276 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:31.899360 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:31.899759 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:31.899851 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:31.902307 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:31.902311 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:31.902311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:31.902328 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:31.904459 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:31.904947 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:31.904951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:31.904951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:31.905091 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:31.905322 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:31.905323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:31.905324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:31.905679 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:31.905843 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:31.905954 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 20:36:31.906105 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
I20260619 20:36:31.906611 28345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:31.908157 28345 catalog_manager.cc:1347] Generated new cluster ID: e91c5e37d494478b9bbce533a77c5b7e
I20260619 20:36:31.908209 28345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:31.924485 28345 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:31.925227 28345 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:31.934904 28345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7793fdbccb3c4a7a8fb5e76e704d4880: Generated new TSK 0
I20260619 20:36:31.935087 28345 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
I20260619 20:36:31.959662 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:40985
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:32.039588 28361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:32.039780 28361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:32.039803 28361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:32.040864 28361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:32.040903 28361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:32.040930 28361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:36:32.042287 28361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:40985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:32.042462 28361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:32.042622 28361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:32.047338 28369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:32.047407 28366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:32.047502 28367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:32.047535 28361 server_base.cc:1034] running on GCE node
I20260619 20:36:32.047762 28361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:32.047950 28361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:32.049120 28361 hybrid_clock.cc:648] HybridClock initialized: now 1781901392049101 us; error 28 us; skew 500 ppm
Jun 19 20:36:32 dist-test-slave-0t40 krb5kdc[28254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901392, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:32.051616 28361 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:36:32.052075 28361 webserver.cc:460] Webserver started at http://127.25.62.129:40791/ using document root <none> and password file <none>
I20260619 20:36:32.052302 28361 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:32.052366 28361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:32.052505 28361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:32.053701 28361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "66627090c260462bb5ba96e56c4dcdb7"
format_stamp: "Formatted at 2026-06-19 20:36:32 on dist-test-slave-0t40"
server_key: "a3a07cc5865595abae21e3c503605813"
server_key_iv: "d11e7ffbe932698b1bd77d8a839fa26f"
server_key_version: "encryptionkey@0"
I20260619 20:36:32.054046 28361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "66627090c260462bb5ba96e56c4dcdb7"
format_stamp: "Formatted at 2026-06-19 20:36:32 on dist-test-slave-0t40"
server_key: "a3a07cc5865595abae21e3c503605813"
server_key_iv: "d11e7ffbe932698b1bd77d8a839fa26f"
server_key_version: "encryptionkey@0"
I20260619 20:36:32.055321 28361 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:36:32.055985 28376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:32.056133 28361 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:32.056226 28361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "66627090c260462bb5ba96e56c4dcdb7"
format_stamp: "Formatted at 2026-06-19 20:36:32 on dist-test-slave-0t40"
server_key: "a3a07cc5865595abae21e3c503605813"
server_key_iv: "d11e7ffbe932698b1bd77d8a839fa26f"
server_key_version: "encryptionkey@0"
I20260619 20:36:32.056294 28361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:32.066372 28361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:32.068607 28361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:32.068781 28361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:32.068992 28361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:32.069304 28361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:32.069352 28361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:32.069392 28361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:32.069415 28361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:32.075003 28361 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:32943
I20260619 20:36:32.075052 28489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:32943 every 8 connection(s)
I20260619 20:36:32.075330 28361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:36:32 dist-test-slave-0t40 krb5kdc[28254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901392, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:32.083899 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:32.075938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:51297 (local address 127.25.62.190:40985)
0619 20:36:32.076011 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:32.076013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:32.076197 (+ 184us) server_negotiation.cc:407] Connection header received
0619 20:36:32.076647 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:32.076648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:32.076658 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:32.076705 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:32.077624 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:32.078113 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:32.079108 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:32.079294 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:32.079343 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:32.081397 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:32.081398 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:32.081398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:32.081405 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:32.082765 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:32.083179 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:32.083180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:32.083180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:32.083229 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:32.083420 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:32.083421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:32.083421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:32.083513 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:32.083561 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:32.083765 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 20:36:32.083833 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 20:36:32.084057 28492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:32.075914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:40985 (local address 127.25.62.129:51297)
0619 20:36:32.076090 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:32.076124 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:32.076570 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:32.076764 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:32.076767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:32.077365 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:32.077551 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:32.077556 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:32.078202 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:32.078203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:32.079034 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:32.079037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:32.079361 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:32.079362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:32.079476 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:32.079983 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:32.079991 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:32.081313 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:32.082902 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:32.082908 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:32.082909 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:32.083113 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:32.083283 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:32.083284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:32.083284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:32.083368 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:32.083563 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:32.083566 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:32.083667 (+ 101us) client_negotiation.cc:769] Sending connection context
0619 20:36:32.083743 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 20:36:32.083849 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":106,"threads_started":1}
I20260619 20:36:32.084080 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 28361
I20260619 20:36:32.084152 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:36:32.084309 25850 external_mini_cluster.cc:1428] Setting key 898a56efac7fbf81840bc9ef294a7239
I20260619 20:36:32.084542 28490 heartbeater.cc:347] Connected to a master server at 127.25.62.190:40985
I20260619 20:36:32.084636 28490 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:32.084810 28490 heartbeater.cc:511] Master 127.25.62.190:40985 requested a full tablet report, sending...
I20260619 20:36:32.085364 28299 ts_manager.cc:194] Registered new tserver with Master: 66627090c260462bb5ba96e56c4dcdb7 (127.25.62.129:32943)
I20260619 20:36:32.086253 28299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:51297
WARNING: no policy specified for kudu/127.25.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
I20260619 20:36:32.108569 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:40985
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:32.188864 28495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:32.189141 28495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:32.189180 28495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:32.190343 28495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:32.190421 28495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:32.190469 28495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:36:32.191942 28495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:40985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:32.192222 28495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:32.192461 28495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:32.197517 28503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:32.197584 28495 server_base.cc:1034] running on GCE node
W20260619 20:36:32.197523 28500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:32.197537 28501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:32.197914 28495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:32.198112 28495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:32.199271 28495 hybrid_clock.cc:648] HybridClock initialized: now 1781901392199251 us; error 31 us; skew 500 ppm
Jun 19 20:36:32 dist-test-slave-0t40 krb5kdc[28254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901392, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:32.201640 28495 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:36:32.202006 28495 webserver.cc:460] Webserver started at http://127.25.62.130:44441/ using document root <none> and password file <none>
I20260619 20:36:32.202198 28495 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:32.202237 28495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:32.202322 28495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:32.203349 28495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "bae6413b19d34dd5a416b745fb7500b4"
format_stamp: "Formatted at 2026-06-19 20:36:32 on dist-test-slave-0t40"
server_key: "6843725a384576f71af789b426702507"
server_key_iv: "f21be6434ee98abe425356a29b465c6a"
server_key_version: "encryptionkey@0"
I20260619 20:36:32.203696 28495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "bae6413b19d34dd5a416b745fb7500b4"
format_stamp: "Formatted at 2026-06-19 20:36:32 on dist-test-slave-0t40"
server_key: "6843725a384576f71af789b426702507"
server_key_iv: "f21be6434ee98abe425356a29b465c6a"
server_key_version: "encryptionkey@0"
I20260619 20:36:32.205152 28495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:36:32.205976 28510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:32.206166 28495 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:32.206247 28495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "bae6413b19d34dd5a416b745fb7500b4"
format_stamp: "Formatted at 2026-06-19 20:36:32 on dist-test-slave-0t40"
server_key: "6843725a384576f71af789b426702507"
server_key_iv: "f21be6434ee98abe425356a29b465c6a"
server_key_version: "encryptionkey@0"
I20260619 20:36:32.206303 28495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:32.224931 28495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:32.227250 28495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:32.227403 28495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:32.227635 28495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:32.227941 28495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:32.227973 28495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:32.228011 28495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:32.228039 28495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:32.233690 28495 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:41701
I20260619 20:36:32.233752 28623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:41701 every 8 connection(s)
I20260619 20:36:32.234077 28495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:36:32 dist-test-slave-0t40 krb5kdc[28254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901392, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:32.243135 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:32.234802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:44593 (local address 127.25.62.190:40985)
0619 20:36:32.234863 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:32.234865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:32.235073 (+ 208us) server_negotiation.cc:407] Connection header received
0619 20:36:32.235593 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:32.235594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:32.235607 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:32.235660 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:32.236596 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:32.237148 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:32.238131 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:32.238327 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:32.238408 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:32.240567 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:32.240568 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:32.240568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:32.240575 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:32.241898 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:32.242307 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:32.242308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:32.242308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:32.242355 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:32.242587 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:32.242588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:32.242588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:32.242685 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:32.242732 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:32.243011 (+ 279us) server_negotiation.cc:299] Negotiation successful
0619 20:36:32.243073 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 20:36:32.243221 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 28495
I20260619 20:36:32.243319 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:36:32.243458 28626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:32.234780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:40985 (local address 127.25.62.130:44593)
0619 20:36:32.234953 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:32.234989 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:32.235491 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:32.235731 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:32.235735 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:32.236290 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:32.236484 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:32.236489 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:32.237235 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:32.237236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:32.238032 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:32.238034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:32.238416 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:32.238418 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:32.238544 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:32.239066 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:32.239074 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:32.240475 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:32.241978 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:32.241985 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:32.241987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:32.242236 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:32.242417 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:32.242417 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:32.242418 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:32.242527 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:32.242738 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:32.242742 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:32.242865 (+ 123us) client_negotiation.cc:769] Sending connection context
0619 20:36:32.242968 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 20:36:32.243175 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
I20260619 20:36:32.243568 25850 external_mini_cluster.cc:1428] Setting key 42695870126f5cdd30dda39e0c5a0f2d
I20260619 20:36:32.244129 28624 heartbeater.cc:347] Connected to a master server at 127.25.62.190:40985
I20260619 20:36:32.244264 28624 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:32.244472 28624 heartbeater.cc:511] Master 127.25.62.190:40985 requested a full tablet report, sending...
I20260619 20:36:32.244926 28299 ts_manager.cc:194] Registered new tserver with Master: bae6413b19d34dd5a416b745fb7500b4 (127.25.62.130:41701)
I20260619 20:36:32.245368 28299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:44593
WARNING: no policy specified for kudu/127.25.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
I20260619 20:36:32.268126 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:40985
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:32.347643 28629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:32.347841 28629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:32.347862 28629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:32.348999 28629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:32.349080 28629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:32.349128 28629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:36:32.350500 28629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:40985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:32.350680 28629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:32.350842 28629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:32.355798 28637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:32.355926 28635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:32.355813 28634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:32.356132 28629 server_base.cc:1034] running on GCE node
I20260619 20:36:32.356335 28629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:32.356612 28629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:32.357762 28629 hybrid_clock.cc:648] HybridClock initialized: now 1781901392357733 us; error 40 us; skew 500 ppm
Jun 19 20:36:32 dist-test-slave-0t40 krb5kdc[28254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901392, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:32.360100 28629 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:36:32.360518 28629 webserver.cc:460] Webserver started at http://127.25.62.131:46407/ using document root <none> and password file <none>
I20260619 20:36:32.360752 28629 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:32.360814 28629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:32.360961 28629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:32.361995 28629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "c13ad0b36c00416b9eb5c2cfdf5d9e36"
format_stamp: "Formatted at 2026-06-19 20:36:32 on dist-test-slave-0t40"
server_key: "ebeca9d0a7ef5f975a8f4ac6d7500001"
server_key_iv: "0245a5bc573a0aadcb5f25db49fa881c"
server_key_version: "encryptionkey@0"
I20260619 20:36:32.362365 28629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "c13ad0b36c00416b9eb5c2cfdf5d9e36"
format_stamp: "Formatted at 2026-06-19 20:36:32 on dist-test-slave-0t40"
server_key: "ebeca9d0a7ef5f975a8f4ac6d7500001"
server_key_iv: "0245a5bc573a0aadcb5f25db49fa881c"
server_key_version: "encryptionkey@0"
I20260619 20:36:32.363634 28629 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:36:32.364341 28644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:32.364557 28629 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:32.364631 28629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "c13ad0b36c00416b9eb5c2cfdf5d9e36"
format_stamp: "Formatted at 2026-06-19 20:36:32 on dist-test-slave-0t40"
server_key: "ebeca9d0a7ef5f975a8f4ac6d7500001"
server_key_iv: "0245a5bc573a0aadcb5f25db49fa881c"
server_key_version: "encryptionkey@0"
I20260619 20:36:32.364683 28629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:32.376600 28629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:32.378811 28629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:32.378930 28629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:32.379185 28629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:32.379597 28629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:32.379653 28629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:32.379693 28629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:32.379715 28629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:32.386888 28629 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:40079
I20260619 20:36:32.386968 28757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:40079 every 8 connection(s)
I20260619 20:36:32.387310 28629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:36:32.392717 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 28629
I20260619 20:36:32.392812 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:36:32.393095 25850 external_mini_cluster.cc:1428] Setting key c1c683fa8dc575bd70a560ecfd7a2a2b
Jun 19 20:36:32 dist-test-slave-0t40 krb5kdc[28254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901392, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:32.396828 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:32.388089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:42973 (local address 127.25.62.190:40985)
0619 20:36:32.388144 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:32.388146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:32.388317 (+ 171us) server_negotiation.cc:407] Connection header received
0619 20:36:32.388804 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:32.388805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:32.388815 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:32.388904 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:32.389874 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:32.390435 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:32.391393 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:32.391602 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:32.391669 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:32.394069 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:32.394070 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:32.394070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:32.394077 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:32.395507 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:32.395909 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:32.395910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:32.395910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:32.395968 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:32.396208 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:32.396209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:32.396210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:32.396325 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:32.396399 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:32.396660 (+ 261us) server_negotiation.cc:299] Negotiation successful
0619 20:36:32.396740 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:36:32.397022 28760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:32.388039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:40985 (local address 127.25.62.131:42973)
0619 20:36:32.388199 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:32.388245 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:32.388719 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:32.388965 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:32.388968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:32.389605 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:32.389796 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:32.389800 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:32.390519 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:32.390520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:32.391313 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:32.391315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:32.391679 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:32.391680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:32.391777 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:32.392291 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:32.392299 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:32.393941 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:32.395604 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:32.395611 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:32.395613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:32.395831 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:32.396032 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:32.396033 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:32.396033 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:32.396142 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:32.396396 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:32.396402 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:32.396536 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 20:36:32.396639 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 20:36:32.396791 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
I20260619 20:36:32.397517 28758 heartbeater.cc:347] Connected to a master server at 127.25.62.190:40985
I20260619 20:36:32.397595 28758 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:32.397778 28758 heartbeater.cc:511] Master 127.25.62.190:40985 requested a full tablet report, sending...
I20260619 20:36:32.398145 28299 ts_manager.cc:194] Registered new tserver with Master: c13ad0b36c00416b9eb5c2cfdf5d9e36 (127.25.62.131:40079)
I20260619 20:36:32.398521 28299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:42973
I20260619 20:36:32.405139 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 20:36:32.416567 28338 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:32.415407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46242 (local address 127.25.62.190:40985)
0619 20:36:32.415488 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:32.415491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:32.415622 (+ 131us) server_negotiation.cc:407] Connection header received
0619 20:36:32.415744 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:32.415745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:32.415763 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:32.415846 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:32.416482 (+ 636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46242: BlockingRecv error: recv got EOF from 127.0.0.1:46242 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 20:36:32.417505 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 28361
I20260619 20:36:32.422254 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 28495
I20260619 20:36:32.426827 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 28629
I20260619 20:36:32.431082 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 28267
2026-06-19T20:36:32Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1815 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:32 dist-test-slave-0t40 krb5kdc[28770](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:32 dist-test-slave-0t40 krb5kdc[28770](info): set up 2 sockets
Jun 19 20:36:32 dist-test-slave-0t40 krb5kdc[28770](info): commencing operation
krb5kdc: starting...
W20260619 20:36:33.469255 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:36:33 dist-test-slave-0t40 krb5kdc[28770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:36:33.583371 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:34099
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:40445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:34099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:33.660636 28783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:33.660861 28783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:33.660897 28783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:33.662058 28783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:33.662122 28783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:33.662145 28783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:33.662166 28783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:33.662194 28783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:33.663661 28783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:40445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:34099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:34099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:33.663913 28783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:33.664119 28783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:33.669001 28788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:33.669026 28789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:33.669093 28783 server_base.cc:1034] running on GCE node
W20260619 20:36:33.668994 28791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:33.669503 28783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:33.669746 28783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:33.670881 28783 hybrid_clock.cc:648] HybridClock initialized: now 1781901393670856 us; error 40 us; skew 500 ppm
Jun 19 20:36:33 dist-test-slave-0t40 krb5kdc[28770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901393, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:33.676472 28783 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:36:33.676900 28783 webserver.cc:460] Webserver started at http://127.25.62.190:44373/ using document root <none> and password file <none>
I20260619 20:36:33.677189 28783 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:33.677237 28783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:33.677371 28783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:33.678395 28783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "891da35180064b5dacfca5fc43d9f754"
format_stamp: "Formatted at 2026-06-19 20:36:33 on dist-test-slave-0t40"
server_key: "924b9b2104626f907d46c421dd827a0c"
server_key_iv: "5d93ba5910ed875c1d56f1aad133866c"
server_key_version: "encryptionkey@0"
I20260619 20:36:33.678761 28783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "891da35180064b5dacfca5fc43d9f754"
format_stamp: "Formatted at 2026-06-19 20:36:33 on dist-test-slave-0t40"
server_key: "924b9b2104626f907d46c421dd827a0c"
server_key_iv: "5d93ba5910ed875c1d56f1aad133866c"
server_key_version: "encryptionkey@0"
I20260619 20:36:33.680025 28783 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:36:33.680686 28798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:33.680845 28783 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:33.680938 28783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "891da35180064b5dacfca5fc43d9f754"
format_stamp: "Formatted at 2026-06-19 20:36:33 on dist-test-slave-0t40"
server_key: "924b9b2104626f907d46c421dd827a0c"
server_key_iv: "5d93ba5910ed875c1d56f1aad133866c"
server_key_version: "encryptionkey@0"
I20260619 20:36:33.681010 28783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:33.696197 28783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:33.698366 28783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:33.698518 28783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:33.702286 28783 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:34099
I20260619 20:36:33.702318 28850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:34099 every 8 connection(s)
I20260619 20:36:33.702761 28783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:33.703405 28851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:33.705770 28851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754: Bootstrap starting.
I20260619 20:36:33.706326 28851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:33.706573 28851 log.cc:824] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:33.707216 28851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754: No bootstrap required, opened a new log
I20260619 20:36:33.707629 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 28783
I20260619 20:36:33.707734 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:36:33.707950 25850 external_mini_cluster.cc:1428] Setting key b861b10b2e4845ba576cee0bf7a85026
I20260619 20:36:33.709982 28851 raft_consensus.cc:348] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "891da35180064b5dacfca5fc43d9f754" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 34099 } }
I20260619 20:36:33.710177 28851 raft_consensus.cc:374] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:33.710222 28851 raft_consensus.cc:729] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 891da35180064b5dacfca5fc43d9f754, State: Initialized, Role: FOLLOWER
I20260619 20:36:33.710363 28851 consensus_queue.cc:260] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "891da35180064b5dacfca5fc43d9f754" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 34099 } }
I20260619 20:36:33.710455 28851 raft_consensus.cc:388] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:33.710491 28851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:33.710546 28851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:33.711211 28851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "891da35180064b5dacfca5fc43d9f754" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 34099 } }
I20260619 20:36:33.711303 28851 leader_election.cc:302] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 891da35180064b5dacfca5fc43d9f754; no voters:
I20260619 20:36:33.711483 28851 leader_election.cc:288] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:33.711513 28856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:33.711621 28856 raft_consensus.cc:686] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754 [term 1 LEADER]: Becoming Leader. State: Replica: 891da35180064b5dacfca5fc43d9f754, State: Running, Role: LEADER
I20260619 20:36:33.711714 28856 consensus_queue.cc:237] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "891da35180064b5dacfca5fc43d9f754" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 34099 } }
I20260619 20:36:33.711740 28851 sys_catalog.cc:564] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:33.712611 28857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "891da35180064b5dacfca5fc43d9f754" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "891da35180064b5dacfca5fc43d9f754" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 34099 } } }
I20260619 20:36:33.712695 28857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:33.712859 28858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 891da35180064b5dacfca5fc43d9f754. Latest consensus state: current_term: 1 leader_uuid: "891da35180064b5dacfca5fc43d9f754" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "891da35180064b5dacfca5fc43d9f754" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 34099 } } }
I20260619 20:36:33.712961 28858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:33.713253 28865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:33.714020 28865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:33.715688 28865 catalog_manager.cc:1347] Generated new cluster ID: d9df35120a5f422f8f1f39c45268638b
I20260619 20:36:33.715747 28865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:36:33 dist-test-slave-0t40 krb5kdc[28770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:33.720480 28855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:33.708442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40182 (local address 127.25.62.190:34099)
0619 20:36:33.708614 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:33.708621 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:33.708646 (+ 25us) server_negotiation.cc:407] Connection header received
0619 20:36:33.709913 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:33.709920 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:33.709971 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:33.710083 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:33.710955 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:33.712166 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:33.713292 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:33.713903 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:33.714015 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:33.716743 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:33.716746 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:33.716747 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:33.716758 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:33.718783 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:33.719192 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:33.719195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:33.719196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:33.719307 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:33.719509 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:33.719510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:33.719510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:33.719814 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:33.720003 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:33.720103 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 20:36:33.720269 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260619 20:36:33.721486 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:36:33.733871 28865 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:33.734612 28865 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:33.737319 28855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:33.733656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40192 (local address 127.25.62.190:34099)
0619 20:36:33.733762 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:33.733765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:33.733928 (+ 163us) server_negotiation.cc:407] Connection header received
0619 20:36:33.734020 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:33.734021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:33.734033 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:33.734113 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:33.734501 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:33.734997 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:33.735749 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:33.735964 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:33.736050 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:33.736769 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:33.736771 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:33.736771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:33.736982 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:33.736999 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:33.737005 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:33.737116 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:33.737189 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 20:36:33.737234 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260619 20:36:33.751219 28865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 891da35180064b5dacfca5fc43d9f754: Generated new TSK 0
I20260619 20:36:33.751415 28865 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:36:33.754341 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 28783
2026-06-19T20:36:33Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1321 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:33 dist-test-slave-0t40 krb5kdc[28884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:33 dist-test-slave-0t40 krb5kdc[28884](info): set up 2 sockets
Jun 19 20:36:33 dist-test-slave-0t40 krb5kdc[28884](info): commencing operation
krb5kdc: starting...
W20260619 20:36:34.793536 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:36:34 dist-test-slave-0t40 krb5kdc[28884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:36:34.910259 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:41967
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:43189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:41967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:34.990042 28897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:34.990243 28897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:34.990267 28897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:34.991429 28897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:34.991475 28897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:34.991492 28897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:34.991506 28897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:34.991546 28897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:34.993000 28897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:43189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:41967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:41967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:34.993307 28897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:34.993546 28897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:36:34.998631 28897 server_base.cc:1034] running on GCE node
W20260619 20:36:34.998567 28902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:34.998581 28903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:34.998595 28905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:34.999158 28897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:34.999439 28897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:35.000631 28897 hybrid_clock.cc:648] HybridClock initialized: now 1781901395000618 us; error 31 us; skew 500 ppm
Jun 19 20:36:34 dist-test-slave-0t40 krb5kdc[28884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901394, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:35.003535 28897 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:36:35.003892 28897 webserver.cc:460] Webserver started at http://127.25.62.190:42313/ using document root <none> and password file <none>
I20260619 20:36:35.004161 28897 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:35.004227 28897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:35.004360 28897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:35.005435 28897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "f5366c1dc85f42b5b7735109454c58a9"
format_stamp: "Formatted at 2026-06-19 20:36:34 on dist-test-slave-0t40"
server_key: "beb00ef099543f97ffd5c2b3b471f2ae"
server_key_iv: "69959aaadfe1bf59ae81dbfe726824be"
server_key_version: "encryptionkey@0"
I20260619 20:36:35.005784 28897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "f5366c1dc85f42b5b7735109454c58a9"
format_stamp: "Formatted at 2026-06-19 20:36:34 on dist-test-slave-0t40"
server_key: "beb00ef099543f97ffd5c2b3b471f2ae"
server_key_iv: "69959aaadfe1bf59ae81dbfe726824be"
server_key_version: "encryptionkey@0"
I20260619 20:36:35.007118 28897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:36:35.007889 28912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:35.008082 28897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:35.008169 28897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "f5366c1dc85f42b5b7735109454c58a9"
format_stamp: "Formatted at 2026-06-19 20:36:34 on dist-test-slave-0t40"
server_key: "beb00ef099543f97ffd5c2b3b471f2ae"
server_key_iv: "69959aaadfe1bf59ae81dbfe726824be"
server_key_version: "encryptionkey@0"
I20260619 20:36:35.008241 28897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:35.022701 28897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:35.024844 28897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:35.024998 28897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:35.029145 28897 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:41967
I20260619 20:36:35.029217 28964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:41967 every 8 connection(s)
I20260619 20:36:35.029537 28897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:35.030454 28965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:35.032734 28965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9: Bootstrap starting.
I20260619 20:36:35.033358 28965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:35.033663 28965 log.cc:824] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:35.034343 28965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9: No bootstrap required, opened a new log
I20260619 20:36:35.034669 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 28897
I20260619 20:36:35.034754 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:36:35.034957 25850 external_mini_cluster.cc:1428] Setting key 949a24dab37e15bdd5ffe8999e5bd884
I20260619 20:36:35.036862 28965 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5366c1dc85f42b5b7735109454c58a9" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41967 } }
I20260619 20:36:35.037040 28965 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:35.037139 28965 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5366c1dc85f42b5b7735109454c58a9, State: Initialized, Role: FOLLOWER
I20260619 20:36:35.037282 28965 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5366c1dc85f42b5b7735109454c58a9" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41967 } }
I20260619 20:36:35.037400 28965 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:35.037449 28965 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:35.037504 28965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:35.038067 28965 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5366c1dc85f42b5b7735109454c58a9" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41967 } }
I20260619 20:36:35.038189 28965 leader_election.cc:302] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5366c1dc85f42b5b7735109454c58a9; no voters:
I20260619 20:36:35.038371 28965 leader_election.cc:288] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:35.038415 28970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:35.038573 28970 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9 [term 1 LEADER]: Becoming Leader. State: Replica: f5366c1dc85f42b5b7735109454c58a9, State: Running, Role: LEADER
I20260619 20:36:35.038673 28965 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:35.038710 28970 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5366c1dc85f42b5b7735109454c58a9" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41967 } }
I20260619 20:36:35.039111 28970 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5366c1dc85f42b5b7735109454c58a9. Latest consensus state: current_term: 1 leader_uuid: "f5366c1dc85f42b5b7735109454c58a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5366c1dc85f42b5b7735109454c58a9" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41967 } } }
I20260619 20:36:35.039178 28970 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:35.039664 28971 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5366c1dc85f42b5b7735109454c58a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5366c1dc85f42b5b7735109454c58a9" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41967 } } }
I20260619 20:36:35.039750 28971 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:35.040033 28979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:35.040570 28979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 20:36:35 dist-test-slave-0t40 krb5kdc[28884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:35.042295 28979 catalog_manager.cc:1347] Generated new cluster ID: 3adf58ee0f7f48af971df02d1521f004
I20260619 20:36:35.042346 28979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:35.046551 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:35.035437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50634 (local address 127.25.62.190:41967)
0619 20:36:35.035663 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:35.035670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:35.035693 (+ 23us) server_negotiation.cc:407] Connection header received
0619 20:36:35.036125 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:35.036131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:35.036174 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:35.036313 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:35.036913 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:35.037976 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:35.038957 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:35.039651 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:35.039781 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:35.042862 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:35.042865 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:35.042866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:35.042877 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:35.044928 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:35.045343 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:35.045346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:35.045347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:35.045446 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:35.045653 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:35.045654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:35.045654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:35.045945 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:35.046107 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:35.046215 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 20:36:35.046345 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":105,"threads_started":1}
I20260619 20:36:35.051854 28979 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:35.052532 28979 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:35.061457 28979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5366c1dc85f42b5b7735109454c58a9: Generated new TSK 0
I20260619 20:36:35.061643 28979 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
I20260619 20:36:35.092526 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41967
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:43189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:35.171928 28991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:35.172122 28991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:35.172144 28991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:35.173213 28991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:35.173244 28991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:35.173270 28991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:36:35.174592 28991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:43189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:35.174778 28991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:35.174939 28991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:35.179692 28999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:35.179805 28996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:35.179862 28991 server_base.cc:1034] running on GCE node
W20260619 20:36:35.179931 28997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:35.180183 28991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:35.180371 28991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:35.181530 28991 hybrid_clock.cc:648] HybridClock initialized: now 1781901395181508 us; error 34 us; skew 500 ppm
Jun 19 20:36:35 dist-test-slave-0t40 krb5kdc[28884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901395, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:35.183899 28991 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:36:35.184365 28991 webserver.cc:460] Webserver started at http://127.25.62.129:43015/ using document root <none> and password file <none>
I20260619 20:36:35.184604 28991 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:35.184674 28991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:35.184823 28991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:35.185904 28991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "88470010fb024542a9c1643718089fc5"
format_stamp: "Formatted at 2026-06-19 20:36:35 on dist-test-slave-0t40"
server_key: "1ba7b9043e7e1470003fb30398582cc0"
server_key_iv: "8964ceaf84e15a9acac3614e451f3d56"
server_key_version: "encryptionkey@0"
I20260619 20:36:35.186285 28991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "88470010fb024542a9c1643718089fc5"
format_stamp: "Formatted at 2026-06-19 20:36:35 on dist-test-slave-0t40"
server_key: "1ba7b9043e7e1470003fb30398582cc0"
server_key_iv: "8964ceaf84e15a9acac3614e451f3d56"
server_key_version: "encryptionkey@0"
I20260619 20:36:35.187633 28991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:36:35.188385 29006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:35.188567 28991 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:35.188664 28991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "88470010fb024542a9c1643718089fc5"
format_stamp: "Formatted at 2026-06-19 20:36:35 on dist-test-slave-0t40"
server_key: "1ba7b9043e7e1470003fb30398582cc0"
server_key_iv: "8964ceaf84e15a9acac3614e451f3d56"
server_key_version: "encryptionkey@0"
I20260619 20:36:35.188748 28991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:35.200875 28991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:35.202803 28991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:35.202971 28991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:35.203197 28991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:35.203505 28991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:35.203576 28991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:35.203617 28991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:35.203648 28991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:35.209645 28991 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:34651
I20260619 20:36:35.209686 29119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:34651 every 8 connection(s)
I20260619 20:36:35.209991 28991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:36:35 dist-test-slave-0t40 krb5kdc[28884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901395, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:35.217231 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 28991
I20260619 20:36:35.217347 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:36:35.217537 25850 external_mini_cluster.cc:1428] Setting key 318d932e14543e5a2a159929b27206ea
I20260619 20:36:35.219086 29122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:35.210625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:41967 (local address 127.25.62.129:48965)
0619 20:36:35.210773 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:35.210812 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:35.211361 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:35.211609 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:35.211612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:35.212197 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:35.212424 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:35.212429 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:35.213130 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:35.213130 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:35.213904 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:35.213906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:35.214256 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:35.214257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:35.214351 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:35.214777 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:35.214784 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:35.216108 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:35.217752 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:35.217757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:35.217759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:35.218031 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:35.218242 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:35.218243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:35.218243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:35.218352 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:35.218623 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:35.218627 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:35.218733 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 20:36:35.218810 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 20:36:35.218934 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260619 20:36:35.219087 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:35.210665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:48965 (local address 127.25.62.190:41967)
0619 20:36:35.210712 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:35.210714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:35.210913 (+ 199us) server_negotiation.cc:407] Connection header received
0619 20:36:35.211482 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:35.211483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:35.211495 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:35.211557 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:35.212511 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:35.213005 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:35.213993 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:35.214190 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:35.214242 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:35.216188 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:35.216189 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:35.216189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:35.216195 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:35.217669 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:35.218111 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:35.218112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:35.218113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:35.218171 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:35.218417 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:35.218418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:35.218418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:35.218560 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:35.218628 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:35.218853 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 20:36:35.218942 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:36:35.219736 29120 heartbeater.cc:347] Connected to a master server at 127.25.62.190:41967
I20260619 20:36:35.219820 29120 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:35.220014 29120 heartbeater.cc:511] Master 127.25.62.190:41967 requested a full tablet report, sending...
I20260619 20:36:35.220512 28929 ts_manager.cc:194] Registered new tserver with Master: 88470010fb024542a9c1643718089fc5 (127.25.62.129:34651)
I20260619 20:36:35.221174 28929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:48965
WARNING: no policy specified for kudu/127.25.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
I20260619 20:36:35.241708 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41967
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:43189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:35.321395 29125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:35.321601 29125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:35.321625 29125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:35.322801 29125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:35.322840 29125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:35.322868 29125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:36:35.324247 29125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:43189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:35.324429 29125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:35.324594 29125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:35.329474 29131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:35.329523 29133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:35.329516 29130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:35.329656 29125 server_base.cc:1034] running on GCE node
I20260619 20:36:35.329907 29125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:35.330168 29125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:35.331362 29125 hybrid_clock.cc:648] HybridClock initialized: now 1781901395331347 us; error 25 us; skew 500 ppm
Jun 19 20:36:35 dist-test-slave-0t40 krb5kdc[28884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901395, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:35.338701 29125 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:36:35.339248 29125 webserver.cc:460] Webserver started at http://127.25.62.130:34949/ using document root <none> and password file <none>
I20260619 20:36:35.339495 29125 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:35.339563 29125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:35.339720 29125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:35.340730 29125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "699ca4a2244f4c878a60f2a80b206cf1"
format_stamp: "Formatted at 2026-06-19 20:36:35 on dist-test-slave-0t40"
server_key: "f202f51423fa00c33c1f2d8f575b1edb"
server_key_iv: "b692e352b03a7aa52dcdf094722682d7"
server_key_version: "encryptionkey@0"
I20260619 20:36:35.341166 29125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "699ca4a2244f4c878a60f2a80b206cf1"
format_stamp: "Formatted at 2026-06-19 20:36:35 on dist-test-slave-0t40"
server_key: "f202f51423fa00c33c1f2d8f575b1edb"
server_key_iv: "b692e352b03a7aa52dcdf094722682d7"
server_key_version: "encryptionkey@0"
I20260619 20:36:35.342465 29125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:35.343187 29140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:35.343384 29125 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:35.343495 29125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "699ca4a2244f4c878a60f2a80b206cf1"
format_stamp: "Formatted at 2026-06-19 20:36:35 on dist-test-slave-0t40"
server_key: "f202f51423fa00c33c1f2d8f575b1edb"
server_key_iv: "b692e352b03a7aa52dcdf094722682d7"
server_key_version: "encryptionkey@0"
I20260619 20:36:35.343585 29125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:35.370010 29125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:35.371964 29125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:35.372119 29125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:35.372345 29125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:35.372658 29125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:35.372709 29125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:35.372746 29125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:35.372773 29125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:35.383023 29125 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:34311
I20260619 20:36:35.383064 29253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:34311 every 8 connection(s)
I20260619 20:36:35.383455 29125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:36:35.386339 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 29125
I20260619 20:36:35.386451 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:36:35.386696 25850 external_mini_cluster.cc:1428] Setting key d828df3e09d02ae9163507a57d7134f1
Jun 19 20:36:35 dist-test-slave-0t40 krb5kdc[28884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901395, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.62.131@KRBTEST.COM; defaulting to no policy
I20260619 20:36:35.399736 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:35.384214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:52887 (local address 127.25.62.190:41967)
0619 20:36:35.384322 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:35.384325 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:35.384510 (+ 185us) server_negotiation.cc:407] Connection header received
0619 20:36:35.385237 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:35.385238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:35.385250 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:35.385305 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:35.386679 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:35.387515 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:35.389270 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:35.389599 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:35.389647 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:35.394588 (+ 4941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:35.394589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:35.394590 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:35.394611 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:35.397886 (+ 3275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:35.398426 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:35.398427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:35.398427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:35.398503 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:35.398802 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:35.398803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:35.398803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:35.398988 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:35.399076 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:35.399502 (+ 426us) server_negotiation.cc:299] Negotiation successful
0619 20:36:35.399617 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 20:36:35.399938 29256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:35.384216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:41967 (local address 127.25.62.130:52887)
0619 20:36:35.384377 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:35.384420 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:35.385144 (+ 724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:35.385376 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:35.385380 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:35.386239 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:35.386576 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:35.386583 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:35.387624 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:35.387626 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:35.389157 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:35.389160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:35.389737 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:35.389738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:35.389880 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:35.392545 (+ 2665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:35.392554 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:35.394502 (+ 1948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:35.397972 (+ 3470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:35.397987 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:35.397989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:35.398326 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:35.398554 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:35.398555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:35.398555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:35.398723 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:35.399110 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:35.399117 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:35.399325 (+ 208us) client_negotiation.cc:769] Sending connection context
0619 20:36:35.399441 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 20:36:35.399689 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
I20260619 20:36:35.400425 29254 heartbeater.cc:347] Connected to a master server at 127.25.62.190:41967
I20260619 20:36:35.400550 29254 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.131@KRBTEST.COM" created.
I20260619 20:36:35.400753 29254 heartbeater.cc:511] Master 127.25.62.190:41967 requested a full tablet report, sending...
I20260619 20:36:35.401225 28929 ts_manager.cc:194] Registered new tserver with Master: 699ca4a2244f4c878a60f2a80b206cf1 (127.25.62.130:34311)
I20260619 20:36:35.401696 28929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:52887
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
I20260619 20:36:35.417707 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41967
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:43189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:35.497876 29259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:35.498064 29259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:35.498086 29259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:35.499209 29259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:35.499248 29259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:35.499298 29259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:36:35.500710 29259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:43189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:35.500936 29259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:35.501181 29259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:35.505844 29267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:35.505890 29265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:35.506016 29264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:35.506124 29259 server_base.cc:1034] running on GCE node
I20260619 20:36:35.506311 29259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:35.506490 29259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:35.507623 29259 hybrid_clock.cc:648] HybridClock initialized: now 1781901395507610 us; error 29 us; skew 500 ppm
Jun 19 20:36:35 dist-test-slave-0t40 krb5kdc[28884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901395, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:35.510212 29259 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:36:35.510607 29259 webserver.cc:460] Webserver started at http://127.25.62.131:46431/ using document root <none> and password file <none>
I20260619 20:36:35.510833 29259 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:35.510896 29259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:35.511049 29259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:35.512081 29259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "309deb317f294dd7a1b4e7a0e30bed19"
format_stamp: "Formatted at 2026-06-19 20:36:35 on dist-test-slave-0t40"
server_key: "c6822176869469a7f8146828de879f9a"
server_key_iv: "fed93c1d349567038ff011a9240e3c2c"
server_key_version: "encryptionkey@0"
I20260619 20:36:35.512432 29259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "309deb317f294dd7a1b4e7a0e30bed19"
format_stamp: "Formatted at 2026-06-19 20:36:35 on dist-test-slave-0t40"
server_key: "c6822176869469a7f8146828de879f9a"
server_key_iv: "fed93c1d349567038ff011a9240e3c2c"
server_key_version: "encryptionkey@0"
I20260619 20:36:35.513803 29259 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:36:35.514511 29274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:35.514679 29259 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:35.514772 29259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "309deb317f294dd7a1b4e7a0e30bed19"
format_stamp: "Formatted at 2026-06-19 20:36:35 on dist-test-slave-0t40"
server_key: "c6822176869469a7f8146828de879f9a"
server_key_iv: "fed93c1d349567038ff011a9240e3c2c"
server_key_version: "encryptionkey@0"
I20260619 20:36:35.514850 29259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:35.530081 29259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:35.532249 29259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:35.532399 29259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:35.532608 29259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:35.532941 29259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:35.533000 29259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:35.533075 29259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:35.533102 29259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:35.538703 29259 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:35421
I20260619 20:36:35.538741 29387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:35421 every 8 connection(s)
I20260619 20:36:35.539077 29259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:36:35.543076 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 29259
I20260619 20:36:35.543160 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:36:35.543330 25850 external_mini_cluster.cc:1428] Setting key eca80b5cacbe438dd23e4202f4adb5b0
Jun 19 20:36:35 dist-test-slave-0t40 krb5kdc[28884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901395, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:35.548492 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:35.539887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:42871 (local address 127.25.62.190:41967)
0619 20:36:35.539928 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:35.539930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:35.540162 (+ 232us) server_negotiation.cc:407] Connection header received
0619 20:36:35.540759 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:35.540760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:35.540773 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:35.540825 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:35.541792 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:35.542297 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:35.543564 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:35.543793 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:35.543860 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:35.546001 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:35.546003 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:35.546003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:35.546009 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:35.547355 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:35.547715 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:35.547716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:35.547716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:35.547763 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:35.547998 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:35.547999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:35.547999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:35.548095 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:35.548144 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:35.548372 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 20:36:35.548426 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:36:35.548733 29390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:35.539898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:41967 (local address 127.25.62.131:42871)
0619 20:36:35.540050 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:35.540082 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:35.540655 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:35.540891 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:35.540894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:35.541515 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:35.541696 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:35.541701 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:35.542390 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:35.542391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:35.543479 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:35.543482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:35.543887 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:35.543889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:35.544033 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:35.544512 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:35.544520 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:35.545910 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:35.547425 (+ 1515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:35.547431 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:35.547433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:35.547652 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:35.547824 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:35.547825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:35.547825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:35.547930 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:35.548152 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:35.548156 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:35.548263 (+ 107us) client_negotiation.cc:769] Sending connection context
0619 20:36:35.548349 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 20:36:35.548476 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260619 20:36:35.549218 29388 heartbeater.cc:347] Connected to a master server at 127.25.62.190:41967
I20260619 20:36:35.549321 29388 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:35.549500 29388 heartbeater.cc:511] Master 127.25.62.190:41967 requested a full tablet report, sending...
I20260619 20:36:35.549887 28929 ts_manager.cc:194] Registered new tserver with Master: 309deb317f294dd7a1b4e7a0e30bed19 (127.25.62.131:35421)
I20260619 20:36:35.550256 28929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:42871
I20260619 20:36:35.554988 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 20:36:35 dist-test-slave-0t40 krb5kdc[28884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901395, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 20:36:35 dist-test-slave-0t40 krb5kdc[28884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901395, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:35.577724 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:35.570311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50640 (local address 127.25.62.190:41967)
0619 20:36:35.570371 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:35.570373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:35.570486 (+ 113us) server_negotiation.cc:407] Connection header received
0619 20:36:35.570570 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:35.570571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:35.570582 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:35.570638 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:35.571201 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:35.571694 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:35.572461 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:35.572653 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:35.572734 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:35.575340 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:35.575341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:35.575342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:35.575350 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:35.576549 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:35.576909 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:35.576910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:35.576910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:35.576967 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:35.577187 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:35.577188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:35.577188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:35.577339 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:35.577447 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:35.577544 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 20:36:35.577623 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260619 20:36:35.577950 28929 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50640
I20260619 20:36:35.578346 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 28991
I20260619 20:36:35.582371 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 29125
I20260619 20:36:35.586634 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 29259
I20260619 20:36:35.590821 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 28897
2026-06-19T20:36:35Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1837 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:35 dist-test-slave-0t40 krb5kdc[29400](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:35 dist-test-slave-0t40 krb5kdc[29400](info): set up 2 sockets
Jun 19 20:36:35 dist-test-slave-0t40 krb5kdc[29400](info): commencing operation
krb5kdc: starting...
W20260619 20:36:36.629323 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:36:36 dist-test-slave-0t40 krb5kdc[29400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:36:36.748369 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:45887
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:45887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:36.825940 29413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:36.826429 29413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:36.826457 29413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:36.827744 29413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:36.827800 29413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:36.827814 29413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:36.827827 29413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:36.827839 29413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:36.829556 29413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:45887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:45887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:36.829798 29413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:36.829998 29413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:36.835009 29418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:36.835220 29413 server_base.cc:1034] running on GCE node
W20260619 20:36:36.835183 29421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:36.835183 29419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:36.835618 29413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:36.835963 29413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:36.837098 29413 hybrid_clock.cc:648] HybridClock initialized: now 1781901396837085 us; error 44 us; skew 500 ppm
Jun 19 20:36:36 dist-test-slave-0t40 krb5kdc[29400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901396, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:36.839591 29413 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:36:36.840019 29413 webserver.cc:460] Webserver started at http://127.25.62.190:39177/ using document root <none> and password file <none>
I20260619 20:36:36.840274 29413 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:36.840343 29413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:36.840495 29413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:36.841650 29413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "be8a11cdc6684968a86c23517e7bf32c"
format_stamp: "Formatted at 2026-06-19 20:36:36 on dist-test-slave-0t40"
server_key: "fc058bf907b52e2a41469d1c52ba4c6c"
server_key_iv: "1006413ab6f5978e519f8a87e7f4098f"
server_key_version: "encryptionkey@0"
I20260619 20:36:36.842020 29413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "be8a11cdc6684968a86c23517e7bf32c"
format_stamp: "Formatted at 2026-06-19 20:36:36 on dist-test-slave-0t40"
server_key: "fc058bf907b52e2a41469d1c52ba4c6c"
server_key_iv: "1006413ab6f5978e519f8a87e7f4098f"
server_key_version: "encryptionkey@0"
I20260619 20:36:36.843369 29413 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:36:36.844132 29428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:36.844301 29413 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:36.844411 29413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "be8a11cdc6684968a86c23517e7bf32c"
format_stamp: "Formatted at 2026-06-19 20:36:36 on dist-test-slave-0t40"
server_key: "fc058bf907b52e2a41469d1c52ba4c6c"
server_key_iv: "1006413ab6f5978e519f8a87e7f4098f"
server_key_version: "encryptionkey@0"
I20260619 20:36:36.844489 29413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:36.858605 29413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:36.860275 29413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:36.860431 29413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:36.864595 29413 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:45887
I20260619 20:36:36.864629 29480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:45887 every 8 connection(s)
I20260619 20:36:36.865226 29413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:36.865859 29481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:36.868057 29481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c: Bootstrap starting.
I20260619 20:36:36.868647 29481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:36.868935 29481 log.cc:824] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:36.869616 29481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c: No bootstrap required, opened a new log
I20260619 20:36:36.871182 29481 raft_consensus.cc:348] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be8a11cdc6684968a86c23517e7bf32c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 45887 } }
I20260619 20:36:36.871315 29481 raft_consensus.cc:374] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:36.871336 29481 raft_consensus.cc:729] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be8a11cdc6684968a86c23517e7bf32c, State: Initialized, Role: FOLLOWER
I20260619 20:36:36.871496 29481 consensus_queue.cc:260] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be8a11cdc6684968a86c23517e7bf32c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 45887 } }
I20260619 20:36:36.871555 29481 raft_consensus.cc:388] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:36.871577 29481 raft_consensus.cc:482] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:36.871629 29481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:36.872483 29481 raft_consensus.cc:504] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be8a11cdc6684968a86c23517e7bf32c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 45887 } }
I20260619 20:36:36.872619 29481 leader_election.cc:302] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be8a11cdc6684968a86c23517e7bf32c; no voters:
I20260619 20:36:36.872690 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 29413
I20260619 20:36:36.872788 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:36:36.872790 29481 leader_election.cc:288] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:36.872857 29484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:36.872987 25850 external_mini_cluster.cc:1428] Setting key d62fa1d32d9f04006b6cb73678906646
I20260619 20:36:36.873055 29484 raft_consensus.cc:686] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c [term 1 LEADER]: Becoming Leader. State: Replica: be8a11cdc6684968a86c23517e7bf32c, State: Running, Role: LEADER
I20260619 20:36:36.873061 29481 sys_catalog.cc:564] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:36.873164 29484 consensus_queue.cc:237] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be8a11cdc6684968a86c23517e7bf32c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 45887 } }
I20260619 20:36:36.874771 29485 sys_catalog.cc:455] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be8a11cdc6684968a86c23517e7bf32c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be8a11cdc6684968a86c23517e7bf32c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 45887 } } }
I20260619 20:36:36.874822 29485 sys_catalog.cc:458] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c [sys.catalog]: This master's current role is: LEADER
W20260619 20:36:36.875073 29500 catalog_manager.cc:1558] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:36:36.875123 29500 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:36:36.874644 29486 sys_catalog.cc:455] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c [sys.catalog]: SysCatalogTable state changed. Reason: New leader be8a11cdc6684968a86c23517e7bf32c. Latest consensus state: current_term: 1 leader_uuid: "be8a11cdc6684968a86c23517e7bf32c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be8a11cdc6684968a86c23517e7bf32c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 45887 } } }
I20260619 20:36:36.875195 29486 sys_catalog.cc:458] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:36.875260 29502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:36.875437 29502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:36.876827 29502 catalog_manager.cc:1347] Generated new cluster ID: 0f1e5243c61b42a6a06f4f3cff7ff247
I20260619 20:36:36.876885 29502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:36:36 dist-test-slave-0t40 krb5kdc[29400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:36.884881 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:36.873607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46188 (local address 127.25.62.190:45887)
0619 20:36:36.873804 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:36.873811 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:36.873838 (+ 27us) server_negotiation.cc:407] Connection header received
0619 20:36:36.873925 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:36.873930 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:36.873975 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:36.874118 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:36.875020 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:36.876396 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:36.877477 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:36.878091 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:36.878170 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:36.881058 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:36.881062 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:36.881062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:36.881075 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:36.883094 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:36.883485 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:36.883488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:36.883488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:36.883612 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:36.883828 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:36.883829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:36.883829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:36.884119 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:36.884261 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:36.884526 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 20:36:36.884680 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":72,"threads_started":1}
I20260619 20:36:36.893476 29502 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:36.894156 29502 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:36.902951 29502 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be8a11cdc6684968a86c23517e7bf32c: Generated new TSK 0
I20260619 20:36:36.903126 29502 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
I20260619 20:36:36.934177 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:45887
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:37.013337 29507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:37.013556 29507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:37.013592 29507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:37.014742 29507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:37.014802 29507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:37.014845 29507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:36:37.016301 29507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:45887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:37.016556 29507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:37.016772 29507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:37.021579 29513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:37.021670 29507 server_base.cc:1034] running on GCE node
W20260619 20:36:37.021517 29512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:37.021517 29515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:37.021987 29507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:37.022238 29507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:37.023450 29507 hybrid_clock.cc:648] HybridClock initialized: now 1781901397023427 us; error 37 us; skew 500 ppm
Jun 19 20:36:37 dist-test-slave-0t40 krb5kdc[29400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901397, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:37.025995 29507 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:36:37.026407 29507 webserver.cc:460] Webserver started at http://127.25.62.129:39525/ using document root <none> and password file <none>
I20260619 20:36:37.026626 29507 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:37.026692 29507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:37.026844 29507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:37.027849 29507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "39c0f2c839c74187850c86c55cb07b88"
format_stamp: "Formatted at 2026-06-19 20:36:37 on dist-test-slave-0t40"
server_key: "2c0c0f2cfdd585bcc422773a6307b021"
server_key_iv: "b062d165cdef6a6712c51ff81c42676a"
server_key_version: "encryptionkey@0"
I20260619 20:36:37.028218 29507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "39c0f2c839c74187850c86c55cb07b88"
format_stamp: "Formatted at 2026-06-19 20:36:37 on dist-test-slave-0t40"
server_key: "2c0c0f2cfdd585bcc422773a6307b021"
server_key_iv: "b062d165cdef6a6712c51ff81c42676a"
server_key_version: "encryptionkey@0"
I20260619 20:36:37.029640 29507 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 20:36:37.030422 29522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:37.030615 29507 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:37.030709 29507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "39c0f2c839c74187850c86c55cb07b88"
format_stamp: "Formatted at 2026-06-19 20:36:37 on dist-test-slave-0t40"
server_key: "2c0c0f2cfdd585bcc422773a6307b021"
server_key_iv: "b062d165cdef6a6712c51ff81c42676a"
server_key_version: "encryptionkey@0"
I20260619 20:36:37.030789 29507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:37.047675 29507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:37.050091 29507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:37.050240 29507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:37.050486 29507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:37.050781 29507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:37.050812 29507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:37.050835 29507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:37.050881 29507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:37.056450 29507 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:36665
I20260619 20:36:37.056526 29635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:36665 every 8 connection(s)
I20260619 20:36:37.056844 29507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:36:37.058827 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 29507
I20260619 20:36:37.058936 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:36:37.059126 25850 external_mini_cluster.cc:1428] Setting key 06262506d7ffaf96ee085d10492d9a0b
Jun 19 20:36:37 dist-test-slave-0t40 krb5kdc[29400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901397, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.62.130@KRBTEST.COM; defaulting to no policy
I20260619 20:36:37.067955 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:37.057634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:34163 (local address 127.25.62.190:45887)
0619 20:36:37.057707 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:37.057710 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:37.057925 (+ 215us) server_negotiation.cc:407] Connection header received
0619 20:36:37.058848 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:37.058850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:37.058866 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:37.058941 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:37.059850 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:37.060621 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:37.062098 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:37.062295 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:37.062354 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:37.064937 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:37.064938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:37.064938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:37.064945 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:37.066433 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:37.066919 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:37.066920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:37.066920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:37.066968 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:37.067245 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:37.067245 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:37.067246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:37.067418 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:37.067508 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:37.067818 (+ 310us) server_negotiation.cc:299] Negotiation successful
0619 20:36:37.067888 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:36:37.068166 29638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:37.057622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:45887 (local address 127.25.62.129:34163)
0619 20:36:37.057799 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:37.057842 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:37.058740 (+ 898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:37.059046 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:37.059051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:37.059400 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:37.059649 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:37.059655 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:37.060720 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:37.060721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:37.062005 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:37.062008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:37.062396 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:37.062397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:37.062547 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:37.063050 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:37.063059 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:37.064839 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:37.066526 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:37.066533 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:37.066535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:37.066840 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:37.067035 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:37.067037 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:37.067037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:37.067176 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:37.067496 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:37.067504 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:37.067671 (+ 167us) client_negotiation.cc:769] Sending connection context
0619 20:36:37.067797 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 20:36:37.067945 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":106,"threads_started":1}
I20260619 20:36:37.068737 29636 heartbeater.cc:347] Connected to a master server at 127.25.62.190:45887
I20260619 20:36:37.068818 29636 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:37.068956 29636 heartbeater.cc:511] Master 127.25.62.190:45887 requested a full tablet report, sending...
I20260619 20:36:37.069458 29445 ts_manager.cc:194] Registered new tserver with Master: 39c0f2c839c74187850c86c55cb07b88 (127.25.62.129:36665)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.130@KRBTEST.COM" created.
I20260619 20:36:37.070060 29445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:34163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
I20260619 20:36:37.084540 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:45887
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:37.165333 29641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:37.165526 29641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:37.165551 29641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:37.166626 29641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:37.166664 29641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:37.166692 29641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:36:37.168123 29641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:45887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:37.168313 29641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:37.168488 29641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:37.173480 29646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:37.173475 29647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:37.173655 29641 server_base.cc:1034] running on GCE node
W20260619 20:36:37.173475 29649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:37.174010 29641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:37.174248 29641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:37.175415 29641 hybrid_clock.cc:648] HybridClock initialized: now 1781901397175409 us; error 39 us; skew 500 ppm
Jun 19 20:36:37 dist-test-slave-0t40 krb5kdc[29400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901397, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:37.178545 29641 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:36:37.178960 29641 webserver.cc:460] Webserver started at http://127.25.62.130:46749/ using document root <none> and password file <none>
I20260619 20:36:37.179150 29641 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:37.179193 29641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:37.179277 29641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:37.180258 29641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "2fa196bee82b4f0ead160a0f2d7500ce"
format_stamp: "Formatted at 2026-06-19 20:36:37 on dist-test-slave-0t40"
server_key: "4915873270cad6e252a4f2f9ab7eb2ed"
server_key_iv: "654e7407636571907d1470e75b7d88fe"
server_key_version: "encryptionkey@0"
I20260619 20:36:37.180541 29641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "2fa196bee82b4f0ead160a0f2d7500ce"
format_stamp: "Formatted at 2026-06-19 20:36:37 on dist-test-slave-0t40"
server_key: "4915873270cad6e252a4f2f9ab7eb2ed"
server_key_iv: "654e7407636571907d1470e75b7d88fe"
server_key_version: "encryptionkey@0"
I20260619 20:36:37.181845 29641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:36:37.182545 29656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:37.182750 29641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:37.182833 29641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "2fa196bee82b4f0ead160a0f2d7500ce"
format_stamp: "Formatted at 2026-06-19 20:36:37 on dist-test-slave-0t40"
server_key: "4915873270cad6e252a4f2f9ab7eb2ed"
server_key_iv: "654e7407636571907d1470e75b7d88fe"
server_key_version: "encryptionkey@0"
I20260619 20:36:37.182893 29641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:37.199590 29641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:37.201610 29641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:37.201722 29641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:37.201915 29641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:37.202229 29641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:37.202268 29641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:37.202291 29641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:37.202337 29641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:37.207916 29641 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:46861
I20260619 20:36:37.207954 29769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:46861 every 8 connection(s)
I20260619 20:36:37.208297 29641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:36:37.209196 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 29641
I20260619 20:36:37.209311 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:36:37.209551 25850 external_mini_cluster.cc:1428] Setting key 633fad185ae0fcc8788ed8d3815498c7
Jun 19 20:36:37 dist-test-slave-0t40 krb5kdc[29400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901397, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.62.131@KRBTEST.COM; defaulting to no policy
I20260619 20:36:37.218923 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:37.209022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:41923 (local address 127.25.62.190:45887)
0619 20:36:37.209217 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:37.209220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:37.209555 (+ 335us) server_negotiation.cc:407] Connection header received
0619 20:36:37.210493 (+ 938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:37.210494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:37.210507 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:37.210567 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:37.211144 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:37.211812 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:37.213128 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:37.213483 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:37.213560 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:37.215730 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:37.215731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:37.215732 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:37.215740 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:37.217415 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:37.217905 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:37.217907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:37.217907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:37.217979 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:37.218289 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:37.218290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:37.218290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:37.218399 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:37.218464 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:37.218790 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 20:36:37.218860 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260619 20:36:37.219118 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:37.209017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:45887 (local address 127.25.62.130:41923)
0619 20:36:37.209418 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:37.209464 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:37.210389 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:37.210621 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:37.210624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:37.210864 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:37.211056 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:37.211061 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:37.211922 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:37.211923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:37.213013 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:37.213016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:37.213571 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:37.213572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:37.213746 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:37.214290 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:37.214298 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:37.215635 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:37.217512 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:37.217518 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:37.217519 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:37.217741 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:37.218057 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:37.218058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:37.218059 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:37.218194 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:37.218534 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:37.218540 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:37.218662 (+ 122us) client_negotiation.cc:769] Sending connection context
0619 20:36:37.218762 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 20:36:37.218890 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":294,"threads_started":1}
I20260619 20:36:37.219636 29770 heartbeater.cc:347] Connected to a master server at 127.25.62.190:45887
I20260619 20:36:37.219748 29770 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:37.219929 29770 heartbeater.cc:511] Master 127.25.62.190:45887 requested a full tablet report, sending...
I20260619 20:36:37.220316 29445 ts_manager.cc:194] Registered new tserver with Master: 2fa196bee82b4f0ead160a0f2d7500ce (127.25.62.130:46861)
I20260619 20:36:37.220641 29445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:41923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
I20260619 20:36:37.236222 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:45887
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:37.315042 29775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:37.315255 29775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:37.315281 29775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:37.316399 29775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:37.316442 29775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:37.316468 29775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:36:37.317999 29775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:45887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:37.318187 29775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:37.318362 29775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:37.323341 29781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:37.323339 29783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:37.323345 29780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:37.323607 29775 server_base.cc:1034] running on GCE node
I20260619 20:36:37.323817 29775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:37.324014 29775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:37.325169 29775 hybrid_clock.cc:648] HybridClock initialized: now 1781901397325156 us; error 30 us; skew 500 ppm
Jun 19 20:36:37 dist-test-slave-0t40 krb5kdc[29400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901397, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:37.327737 29775 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:36:37.328145 29775 webserver.cc:460] Webserver started at http://127.25.62.131:43135/ using document root <none> and password file <none>
I20260619 20:36:37.328342 29775 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:37.328397 29775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:37.328495 29775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:37.329594 29775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "9e9d6c8e21bd4f78b044a249900fce00"
format_stamp: "Formatted at 2026-06-19 20:36:37 on dist-test-slave-0t40"
server_key: "18bec777cad0d1b2c63c692432b3600a"
server_key_iv: "f18e3f000c3677174634da43b106f2c6"
server_key_version: "encryptionkey@0"
I20260619 20:36:37.329917 29775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "9e9d6c8e21bd4f78b044a249900fce00"
format_stamp: "Formatted at 2026-06-19 20:36:37 on dist-test-slave-0t40"
server_key: "18bec777cad0d1b2c63c692432b3600a"
server_key_iv: "f18e3f000c3677174634da43b106f2c6"
server_key_version: "encryptionkey@0"
I20260619 20:36:37.331318 29775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:36:37.332028 29790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:37.332193 29775 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:37.332273 29775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "9e9d6c8e21bd4f78b044a249900fce00"
format_stamp: "Formatted at 2026-06-19 20:36:37 on dist-test-slave-0t40"
server_key: "18bec777cad0d1b2c63c692432b3600a"
server_key_iv: "f18e3f000c3677174634da43b106f2c6"
server_key_version: "encryptionkey@0"
I20260619 20:36:37.332325 29775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:37.347599 29775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:37.349483 29775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:37.349591 29775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:37.349754 29775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:37.349996 29775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:37.350025 29775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:37.350046 29775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:37.350059 29775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:37.355491 29775 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:35621
I20260619 20:36:37.355574 29903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:35621 every 8 connection(s)
I20260619 20:36:37.355873 29775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:36:37.360788 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 29775
I20260619 20:36:37.360880 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:36:37.361138 25850 external_mini_cluster.cc:1428] Setting key 3294ed5de0fafb98ec16430e18994a20
Jun 19 20:36:37 dist-test-slave-0t40 krb5kdc[29400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901397, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:37.365801 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:37.356604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:48841 (local address 127.25.62.190:45887)
0619 20:36:37.356689 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:37.356692 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:37.356875 (+ 183us) server_negotiation.cc:407] Connection header received
0619 20:36:37.357866 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:37.357868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:37.357882 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:37.357930 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:37.358565 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:37.359077 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:37.360100 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:37.360311 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:37.360410 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:37.362830 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:37.362832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:37.362832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:37.362840 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:37.364286 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:37.364752 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:37.364753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:37.364754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:37.364816 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:37.365123 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:37.365124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:37.365124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:37.365262 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:37.365345 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:37.365625 (+ 280us) server_negotiation.cc:299] Negotiation successful
0619 20:36:37.365709 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:36:37.365993 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:37.356567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:45887 (local address 127.25.62.131:48841)
0619 20:36:37.356751 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:37.356786 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:37.357745 (+ 959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:37.358016 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:37.358020 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:37.358267 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:37.358453 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:37.358457 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:37.359188 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:37.359189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:37.359995 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:37.359997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:37.360419 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:37.360420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:37.360544 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:37.361105 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:37.361113 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:37.362715 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:37.364417 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:37.364424 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:37.364426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:37.364656 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:37.364888 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:37.364889 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:37.364889 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:37.365006 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:37.365335 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:37.365341 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:37.365482 (+ 141us) client_negotiation.cc:769] Sending connection context
0619 20:36:37.365587 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 20:36:37.365739 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":94,"threads_started":1}
I20260619 20:36:37.366415 29904 heartbeater.cc:347] Connected to a master server at 127.25.62.190:45887
I20260619 20:36:37.366506 29904 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:37.366639 29904 heartbeater.cc:511] Master 127.25.62.190:45887 requested a full tablet report, sending...
I20260619 20:36:37.366950 29445 ts_manager.cc:194] Registered new tserver with Master: 9e9d6c8e21bd4f78b044a249900fce00 (127.25.62.131:35621)
I20260619 20:36:37.367275 29445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:48841
I20260619 20:36:37.372929 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 20:36:37 dist-test-slave-0t40 krb5kdc[29400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 20:36:37 dist-test-slave-0t40 krb5kdc[29400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.129@KRBTEST.COM
I20260619 20:36:37.397620 29911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:37.389680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39856 (local address 127.25.62.129:36665)
0619 20:36:37.389855 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:37.389868 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:37.389883 (+ 15us) server_negotiation.cc:407] Connection header received
0619 20:36:37.389912 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:37.389914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:37.389929 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:37.390004 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:37.390536 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:37.391270 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:37.392047 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:37.392400 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:37.392479 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:37.394823 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:37.394832 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:37.394832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:37.394839 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:37.396466 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:37.396804 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:37.396807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:37.396807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:37.396859 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:37.397073 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:37.397074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:37.397074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:37.397220 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:37.397329 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:37.397451 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 20:36:37.397512 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
Jun 19 20:36:37 dist-test-slave-0t40 krb5kdc[29400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:37.405607 29489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:37.398965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46204 (local address 127.25.62.190:45887)
0619 20:36:37.399033 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:37.399035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:37.399046 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:36:37.399121 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:37.399122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:37.399132 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:37.399186 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:37.399621 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:37.400112 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:37.400750 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:37.400938 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:37.400994 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:37.403139 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:37.403140 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:37.403141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:37.403147 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:37.404472 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:37.404813 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:37.404814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:37.404814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:37.404874 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:37.405128 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:37.405128 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:37.405129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:37.405269 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:37.405349 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:37.405483 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 20:36:37.405545 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260619 20:36:37.405869 29445 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46204
I20260619 20:36:37.406203 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 29507
I20260619 20:36:37.411391 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 29641
I20260619 20:36:37.415616 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 29775
I20260619 20:36:37.419466 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 29413
2026-06-19T20:36:37Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1829 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:37 dist-test-slave-0t40 krb5kdc[29916](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:37 dist-test-slave-0t40 krb5kdc[29916](info): set up 2 sockets
Jun 19 20:36:37 dist-test-slave-0t40 krb5kdc[29916](info): commencing operation
krb5kdc: starting...
W20260619 20:36:38.453336 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:36:38 dist-test-slave-0t40 krb5kdc[29916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:36:38.572616 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:32907
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:32907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:38.650902 29929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:38.651144 29929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:38.651180 29929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:38.652325 29929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:38.652391 29929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:38.652415 29929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:38.652443 29929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:38.652474 29929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:38.654006 29929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:32907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:32907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:38.654268 29929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:38.654481 29929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:38.659188 29934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:38.659166 29937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:38.659209 29935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:38.659262 29929 server_base.cc:1034] running on GCE node
I20260619 20:36:38.659595 29929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:38.659878 29929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:38.661063 29929 hybrid_clock.cc:648] HybridClock initialized: now 1781901398661034 us; error 44 us; skew 500 ppm
Jun 19 20:36:38 dist-test-slave-0t40 krb5kdc[29916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901398, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:38.663843 29929 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:36:38.664219 29929 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:38.664290 29929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:38.664425 29929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:38.665657 29929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "4eb4edcd52e8487c849f1cdddd312e71"
format_stamp: "Formatted at 2026-06-19 20:36:38 on dist-test-slave-0t40"
server_key: "da17d0d7203b9644d2a8355b9c10b6bc"
server_key_iv: "b9ea9e101adf141ee46b2aecf1e8e919"
server_key_version: "encryptionkey@0"
I20260619 20:36:38.666096 29929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "4eb4edcd52e8487c849f1cdddd312e71"
format_stamp: "Formatted at 2026-06-19 20:36:38 on dist-test-slave-0t40"
server_key: "da17d0d7203b9644d2a8355b9c10b6bc"
server_key_iv: "b9ea9e101adf141ee46b2aecf1e8e919"
server_key_version: "encryptionkey@0"
I20260619 20:36:38.667968 29929 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 20:36:38.668825 29943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:38.669016 29929 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:38.669121 29929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "4eb4edcd52e8487c849f1cdddd312e71"
format_stamp: "Formatted at 2026-06-19 20:36:38 on dist-test-slave-0t40"
server_key: "da17d0d7203b9644d2a8355b9c10b6bc"
server_key_iv: "b9ea9e101adf141ee46b2aecf1e8e919"
server_key_version: "encryptionkey@0"
I20260619 20:36:38.669212 29929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:38.683820 29929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:38.686007 29929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:38.686157 29929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:38.689930 29929 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:32907
I20260619 20:36:38.689961 29995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:32907 every 8 connection(s)
I20260619 20:36:38.690245 29929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:38.691126 29996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:38.693310 29996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71: Bootstrap starting.
I20260619 20:36:38.693852 29996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:38.694141 29996 log.cc:824] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:38.694785 29996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71: No bootstrap required, opened a new log
I20260619 20:36:38.696297 29996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb4edcd52e8487c849f1cdddd312e71" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 32907 } }
I20260619 20:36:38.696424 29996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:38.696444 29996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eb4edcd52e8487c849f1cdddd312e71, State: Initialized, Role: FOLLOWER
I20260619 20:36:38.696607 29996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb4edcd52e8487c849f1cdddd312e71" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 32907 } }
I20260619 20:36:38.696667 29996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:38.696688 29996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:38.696733 29996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:38.697027 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 29929
I20260619 20:36:38.697147 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:36:38.697320 25850 external_mini_cluster.cc:1428] Setting key f03dfafd0a11bc6ef8821f71b63a9c96
I20260619 20:36:38.697820 29996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb4edcd52e8487c849f1cdddd312e71" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 32907 } }
I20260619 20:36:38.697930 29996 leader_election.cc:302] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4eb4edcd52e8487c849f1cdddd312e71; no voters:
I20260619 20:36:38.698107 29996 leader_election.cc:288] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:38.698141 30001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:38.698276 30001 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71 [term 1 LEADER]: Becoming Leader. State: Replica: 4eb4edcd52e8487c849f1cdddd312e71, State: Running, Role: LEADER
I20260619 20:36:38.698354 29996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:38.699209 30001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb4edcd52e8487c849f1cdddd312e71" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 32907 } }
I20260619 20:36:38.699873 30002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4eb4edcd52e8487c849f1cdddd312e71. Latest consensus state: current_term: 1 leader_uuid: "4eb4edcd52e8487c849f1cdddd312e71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb4edcd52e8487c849f1cdddd312e71" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 32907 } } }
I20260619 20:36:38.701102 30002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:38.699915 30003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4eb4edcd52e8487c849f1cdddd312e71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4eb4edcd52e8487c849f1cdddd312e71" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 32907 } } }
I20260619 20:36:38.701202 30003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71 [sys.catalog]: This master's current role is: LEADER
W20260619 20:36:38.700389 30015 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:36:38.701292 30015 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:36:38.701411 30018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:38.701596 30018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:38.703130 30018 catalog_manager.cc:1347] Generated new cluster ID: fc205f3c88e445c9955204533524144f
I20260619 20:36:38.703181 30018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:36:38 dist-test-slave-0t40 krb5kdc[29916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:38.709233 30000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:38.697810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52688 (local address 127.25.62.190:32907)
0619 20:36:38.698016 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:38.698040 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:38.698073 (+ 33us) server_negotiation.cc:407] Connection header received
0619 20:36:38.698192 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:38.698199 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:38.698260 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:38.698396 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:38.699191 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:38.700831 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:38.701888 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:38.702427 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:38.702536 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:38.705226 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:38.705230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:38.705230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:38.705241 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:38.707226 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:38.707607 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:38.707610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:38.707611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:38.707728 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:38.707922 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:38.707922 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:38.707923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:38.708224 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:38.708370 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:38.708865 (+ 495us) server_negotiation.cc:299] Negotiation successful
0619 20:36:38.709003 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":84,"threads_started":1}
I20260619 20:36:38.718412 30018 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:38.719520 30018 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:38.725644 30018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71: Generated new TSK 0
I20260619 20:36:38.725827 30018 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
I20260619 20:36:38.754099 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:32907
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:38.833830 30022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:38.834051 30022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:38.834091 30022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:38.835237 30022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:38.835297 30022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:38.835340 30022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:36:38.836804 30022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:32907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:38.837114 30022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:38.837334 30022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:38.842218 30028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:38.842276 30022 server_base.cc:1034] running on GCE node
W20260619 20:36:38.842329 30027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:38.842424 30030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:38.842659 30022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:38.842933 30022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:38.844116 30022 hybrid_clock.cc:648] HybridClock initialized: now 1781901398844107 us; error 37 us; skew 500 ppm
Jun 19 20:36:38 dist-test-slave-0t40 krb5kdc[29916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901398, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:38.846576 30022 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:36:38.846912 30022 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:38.846985 30022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:38.847128 30022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:38.848129 30022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "7c2c664c3b4047e4ac4382641e5c2bde"
format_stamp: "Formatted at 2026-06-19 20:36:38 on dist-test-slave-0t40"
server_key: "bbf8fca8ae6c3a448b7d2608552f7ba7"
server_key_iv: "a411fa3649fec59cc2eb5e433fb2ba6e"
server_key_version: "encryptionkey@0"
I20260619 20:36:38.848482 30022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "7c2c664c3b4047e4ac4382641e5c2bde"
format_stamp: "Formatted at 2026-06-19 20:36:38 on dist-test-slave-0t40"
server_key: "bbf8fca8ae6c3a448b7d2608552f7ba7"
server_key_iv: "a411fa3649fec59cc2eb5e433fb2ba6e"
server_key_version: "encryptionkey@0"
I20260619 20:36:38.849823 30022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:36:38.850478 30036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:38.850648 30022 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:38.850754 30022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "7c2c664c3b4047e4ac4382641e5c2bde"
format_stamp: "Formatted at 2026-06-19 20:36:38 on dist-test-slave-0t40"
server_key: "bbf8fca8ae6c3a448b7d2608552f7ba7"
server_key_iv: "a411fa3649fec59cc2eb5e433fb2ba6e"
server_key_version: "encryptionkey@0"
I20260619 20:36:38.850822 30022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:38.875224 30022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:38.877172 30022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:38.877331 30022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:38.877569 30022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:38.877873 30022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:38.877930 30022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:38.877974 30022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:38.878003 30022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:38.883888 30022 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:46793
I20260619 20:36:38.883934 30149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:46793 every 8 connection(s)
I20260619 20:36:38.884181 30022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:36:38.889258 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 30022
I20260619 20:36:38.889360 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:36:38.889571 25850 external_mini_cluster.cc:1428] Setting key 91d2d6828446106ea1570c227f05518d
Jun 19 20:36:38 dist-test-slave-0t40 krb5kdc[29916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901398, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:38.894299 30000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:38.884895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:60287 (local address 127.25.62.190:32907)
0619 20:36:38.884941 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:38.884943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:38.885328 (+ 385us) server_negotiation.cc:407] Connection header received
0619 20:36:38.885878 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:38.885880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:38.885896 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:38.885954 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:38.886907 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:38.887414 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:38.888452 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:38.888658 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:38.888729 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:38.891058 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:38.891060 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:38.891060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:38.891071 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:38.892763 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:38.893264 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:38.893266 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:38.893266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:38.893336 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:38.893589 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:38.893590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:38.893591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:38.893744 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:38.893821 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:38.894103 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 20:36:38.894195 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:36:38.894307 30152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:38.884869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:32907 (local address 127.25.62.129:60287)
0619 20:36:38.885061 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:38.885177 (+ 116us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:38.885763 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:38.886030 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:38.886034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:38.886295 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:38.886798 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:38.886804 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:38.887521 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:38.887522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:38.888335 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:38.888337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:38.888750 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:38.888751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:38.888871 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:38.889441 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:38.889452 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:38.890957 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:38.892860 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:38.892866 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:38.892868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:38.893188 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:38.893408 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:38.893409 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:38.893409 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:38.893510 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:38.893814 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:38.893818 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:38.893955 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 20:36:38.894034 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 20:36:38.894152 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":79,"threads_started":1}
I20260619 20:36:38.895072 30150 heartbeater.cc:347] Connected to a master server at 127.25.62.190:32907
I20260619 20:36:38.895184 30150 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:38.895506 30150 heartbeater.cc:511] Master 127.25.62.190:32907 requested a full tablet report, sending...
I20260619 20:36:38.896147 29960 ts_manager.cc:194] Registered new tserver with Master: 7c2c664c3b4047e4ac4382641e5c2bde (127.25.62.129:46793)
I20260619 20:36:38.897141 29960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:60287
WARNING: no policy specified for kudu/127.25.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
I20260619 20:36:38.915647 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:32907
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:38.994378 30155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:38.994594 30155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:38.994637 30155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:38.995811 30155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:38.995858 30155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:38.995919 30155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:36:38.997418 30155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:32907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:38.997663 30155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:38.997896 30155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:39.003403 30163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:39.003434 30161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:39.003463 30155 server_base.cc:1034] running on GCE node
W20260619 20:36:39.003434 30160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:39.003937 30155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:39.004208 30155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:39.005380 30155 hybrid_clock.cc:648] HybridClock initialized: now 1781901399005364 us; error 34 us; skew 500 ppm
Jun 19 20:36:39 dist-test-slave-0t40 krb5kdc[29916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901399, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:39.007915 30155 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:36:39.008270 30155 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:39.008319 30155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:39.008415 30155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:39.009462 30155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "e29d76830c1d4ba385928b92fa6e4c75"
format_stamp: "Formatted at 2026-06-19 20:36:39 on dist-test-slave-0t40"
server_key: "55fecde3403ec3226795d86edc91b9f7"
server_key_iv: "480f34043503c6a5ecf502fb238591a3"
server_key_version: "encryptionkey@0"
I20260619 20:36:39.009816 30155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "e29d76830c1d4ba385928b92fa6e4c75"
format_stamp: "Formatted at 2026-06-19 20:36:39 on dist-test-slave-0t40"
server_key: "55fecde3403ec3226795d86edc91b9f7"
server_key_iv: "480f34043503c6a5ecf502fb238591a3"
server_key_version: "encryptionkey@0"
I20260619 20:36:39.011160 30155 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:36:39.011996 30169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:39.012208 30155 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:39.012283 30155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "e29d76830c1d4ba385928b92fa6e4c75"
format_stamp: "Formatted at 2026-06-19 20:36:39 on dist-test-slave-0t40"
server_key: "55fecde3403ec3226795d86edc91b9f7"
server_key_iv: "480f34043503c6a5ecf502fb238591a3"
server_key_version: "encryptionkey@0"
I20260619 20:36:39.012377 30155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:39.029488 30155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:39.031795 30155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:39.031940 30155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:39.032188 30155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:39.032509 30155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:39.032544 30155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:39.032586 30155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:39.032613 30155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:39.038455 30155 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:37335
I20260619 20:36:39.038496 30282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:37335 every 8 connection(s)
I20260619 20:36:39.038786 30155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:36:39.040182 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 30155
I20260619 20:36:39.040266 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:36:39.040491 25850 external_mini_cluster.cc:1428] Setting key 7fd4e7c96a14e9084dbff244f6bb93dd
Jun 19 20:36:39 dist-test-slave-0t40 krb5kdc[29916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901399, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.62.131@KRBTEST.COM; defaulting to no policy
I20260619 20:36:39.049365 30000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.039517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:60255 (local address 127.25.62.190:32907)
0619 20:36:39.039564 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:39.039566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:39.039761 (+ 195us) server_negotiation.cc:407] Connection header received
0619 20:36:39.040326 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:39.040328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:39.040341 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:39.040429 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:39.041665 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.042463 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.043539 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.043860 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.043942 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:39.046265 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:39.046266 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:39.046267 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:39.046276 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:39.047920 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:39.048310 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:39.048311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:39.048312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:39.048368 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:39.048614 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:39.048615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:39.048616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:39.048752 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:39.048823 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:39.049198 (+ 375us) server_negotiation.cc:299] Negotiation successful
0619 20:36:39.049293 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:36:39.049491 30285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.039475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:32907 (local address 127.25.62.130:60255)
0619 20:36:39.039627 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:39.039668 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:39.040199 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:39.040492 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:39.040495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:39.040889 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:39.041571 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:39.041577 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.042602 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.042603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:39.043445 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:39.043447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.043960 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.043960 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:39.044063 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:39.044630 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:39.044639 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:39.046172 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:39.047998 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:39.048004 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:39.048006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:39.048228 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:39.048442 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:39.048443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:39.048443 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:39.048543 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:39.048817 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:39.048820 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:39.048997 (+ 177us) client_negotiation.cc:769] Sending connection context
0619 20:36:39.049157 (+ 160us) client_negotiation.cc:240] Negotiation successful
0619 20:36:39.049284 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260619 20:36:39.050014 30283 heartbeater.cc:347] Connected to a master server at 127.25.62.190:32907
I20260619 20:36:39.050134 30283 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:39.050285 30283 heartbeater.cc:511] Master 127.25.62.190:32907 requested a full tablet report, sending...
I20260619 20:36:39.050676 29960 ts_manager.cc:194] Registered new tserver with Master: e29d76830c1d4ba385928b92fa6e4c75 (127.25.62.130:37335)
I20260619 20:36:39.051158 29960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:60255
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
I20260619 20:36:39.067520 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:32907
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:39.147367 30288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:39.147606 30288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:39.147643 30288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:39.148787 30288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:39.148850 30288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:39.148895 30288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:36:39.150455 30288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:32907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:39.150713 30288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:39.150939 30288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:39.155941 30293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:39.156008 30294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:39.156033 30296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:39.156018 30288 server_base.cc:1034] running on GCE node
I20260619 20:36:39.156414 30288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:39.156667 30288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:39.157825 30288 hybrid_clock.cc:648] HybridClock initialized: now 1781901399157821 us; error 35 us; skew 500 ppm
Jun 19 20:36:39 dist-test-slave-0t40 krb5kdc[29916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901399, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:39.160298 30288 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:36:39.160670 30288 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:39.160718 30288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:39.160806 30288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:39.161837 30288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "fe0fb72cca7e47828934d28edb1df76b"
format_stamp: "Formatted at 2026-06-19 20:36:39 on dist-test-slave-0t40"
server_key: "64990a5195b803067033a3731b1765fa"
server_key_iv: "c3a7e4882891b6fb4a1fab855b976cc9"
server_key_version: "encryptionkey@0"
I20260619 20:36:39.162132 30288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "fe0fb72cca7e47828934d28edb1df76b"
format_stamp: "Formatted at 2026-06-19 20:36:39 on dist-test-slave-0t40"
server_key: "64990a5195b803067033a3731b1765fa"
server_key_iv: "c3a7e4882891b6fb4a1fab855b976cc9"
server_key_version: "encryptionkey@0"
I20260619 20:36:39.163295 30288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:36:39.163939 30302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:39.164116 30288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:36:39.164177 30288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "fe0fb72cca7e47828934d28edb1df76b"
format_stamp: "Formatted at 2026-06-19 20:36:39 on dist-test-slave-0t40"
server_key: "64990a5195b803067033a3731b1765fa"
server_key_iv: "c3a7e4882891b6fb4a1fab855b976cc9"
server_key_version: "encryptionkey@0"
I20260619 20:36:39.164227 30288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:39.184166 30288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:39.186499 30288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:39.186621 30288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:39.186815 30288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:39.187183 30288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:39.187223 30288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:39.187247 30288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:39.187258 30288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:39.193099 30288 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:36413
I20260619 20:36:39.193142 30415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:36413 every 8 connection(s)
I20260619 20:36:39.193393 30288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:36:39 dist-test-slave-0t40 krb5kdc[29916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901399, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:39.202304 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 30288
I20260619 20:36:39.202442 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:36:39.202687 25850 external_mini_cluster.cc:1428] Setting key 4eb3207bbf92292c5a198959313d4fd0
I20260619 20:36:39.202906 30000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.194066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:47653 (local address 127.25.62.190:32907)
0619 20:36:39.194136 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:39.194138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:39.194338 (+ 200us) server_negotiation.cc:407] Connection header received
0619 20:36:39.194908 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:39.194909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:39.194919 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:39.194981 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:39.195906 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.196413 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.197453 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.197641 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.197704 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:39.199993 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:39.199994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:39.199995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:39.200000 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:39.201478 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:39.201833 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:39.201834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:39.201834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:39.201904 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:39.202158 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:39.202159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:39.202159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:39.202309 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:39.202392 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:39.202705 (+ 313us) server_negotiation.cc:299] Negotiation successful
0619 20:36:39.202791 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 20:36:39.203094 30418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.194021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:32907 (local address 127.25.62.131:47653)
0619 20:36:39.194193 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:39.194232 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:39.194791 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:39.195060 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:39.195064 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:39.195308 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:39.195784 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:39.195789 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.196497 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.196498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:39.197373 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:39.197376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.197716 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.197716 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:39.197809 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:39.198337 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:39.198346 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:39.199913 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:39.201555 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:39.201560 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:39.201562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:39.201766 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:39.201962 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:39.201963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:39.201963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:39.202065 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:39.202401 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:39.202405 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:39.202574 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 20:36:39.202659 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 20:36:39.202815 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":105,"threads_started":1}
I20260619 20:36:39.203723 30416 heartbeater.cc:347] Connected to a master server at 127.25.62.190:32907
I20260619 20:36:39.203822 30416 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:39.203975 30416 heartbeater.cc:511] Master 127.25.62.190:32907 requested a full tablet report, sending...
I20260619 20:36:39.204401 29960 ts_manager.cc:194] Registered new tserver with Master: fe0fb72cca7e47828934d28edb1df76b (127.25.62.131:36413)
I20260619 20:36:39.204877 29960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:47653
I20260619 20:36:39.214581 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:36:39.221486 30000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.215842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52704 (local address 127.25.62.190:32907)
0619 20:36:39.215900 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:39.215902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:39.216006 (+ 104us) server_negotiation.cc:407] Connection header received
0619 20:36:39.216058 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:39.216058 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:39.216067 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:39.216133 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:39.216716 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.217244 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.217901 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.218073 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.218121 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:39.219229 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:39.219229 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:39.219230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:39.219234 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:39.220351 (+ 1117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:39.220661 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:39.220662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:39.220662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:39.220717 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:39.220909 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:39.220910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:39.220910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:39.221143 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:39.221231 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:39.221314 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 20:36:39.221376 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 20:36:39.223140 29960 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:36:39.224680 29960 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:36:39.231240 30425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.227121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54910 (local address 127.25.62.129:46793)
0619 20:36:39.227586 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:39.227590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:39.227601 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:36:39.227646 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:39.227648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:39.227661 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:39.227757 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:39.227931 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.229102 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.230363 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.230851 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.230910 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:39.230948 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:39.231112 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 20:36:39.231157 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":80,"threads_started":1}
I20260619 20:36:39.231225 30426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.227002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:46793 (local address 127.0.0.1:54910)
0619 20:36:39.227313 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:39.227356 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:39.227431 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:39.227766 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:39.227770 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:39.227772 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:39.227842 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:39.227844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.229194 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.229195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:39.230287 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:39.230290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.230934 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.230935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:39.230996 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:39.231008 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 20:36:39.231092 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 20:36:39.231148 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":1,"thread_start_us":54,"threads_started":1}
I20260619 20:36:39.231878 30427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.227351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:36413 (local address 127.0.0.1:49924)
0619 20:36:39.227858 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:39.227868 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:39.227915 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:39.228125 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:39.228126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:39.228128 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:39.228159 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:39.228160 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.229355 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.229356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:39.230240 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:39.230242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.231641 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.231642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:39.231706 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:39.231710 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:39.231768 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 20:36:39.231809 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":2,"thread_start_us":46,"threads_started":1}
I20260619 20:36:39.231930 30429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.227435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49924 (local address 127.25.62.131:36413)
0619 20:36:39.227588 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:39.227592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:39.227971 (+ 379us) server_negotiation.cc:407] Connection header received
0619 20:36:39.228010 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:39.228012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:39.228027 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:39.228111 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:39.228220 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.229250 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.230971 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.231545 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.231638 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:39.231690 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:39.231799 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 20:36:39.231844 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260619 20:36:39.232023 30084 tablet_service.cc:1467] Processing CreateTablet for tablet 3b7a94de04be440795495e93183ad64e (DEFAULT_TABLE table=test-table [id=91afc70895dc4643851a29f188d70e5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:39.232416 30084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b7a94de04be440795495e93183ad64e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:39.232792 30350 tablet_service.cc:1467] Processing CreateTablet for tablet 3b7a94de04be440795495e93183ad64e (DEFAULT_TABLE table=test-table [id=91afc70895dc4643851a29f188d70e5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:39.232986 30428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.227322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60774 (local address 127.25.62.130:37335)
0619 20:36:39.227555 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:39.227560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:39.227570 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:36:39.227610 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:39.227612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:39.227627 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:39.227750 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:39.228004 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.228797 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.231728 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.232610 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.232687 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:39.232738 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:39.232866 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 20:36:39.232907 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":89,"threads_started":1}
I20260619 20:36:39.232956 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.227203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:37335 (local address 127.0.0.1:60774)
0619 20:36:39.227440 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:39.227451 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:39.227501 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:39.227835 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:39.227836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:39.227839 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:39.227921 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:39.227923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.228892 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.228894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:39.230288 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:39.230290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.232699 (+ 2409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.232700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:39.232764 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:39.232768 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:39.232839 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 20:36:39.232879 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":5,"thread_start_us":51,"threads_started":1}
I20260619 20:36:39.233175 30350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b7a94de04be440795495e93183ad64e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:39.235260 30430 tablet_bootstrap.cc:492] T 3b7a94de04be440795495e93183ad64e P 7c2c664c3b4047e4ac4382641e5c2bde: Bootstrap starting.
I20260619 20:36:39.235689 30431 tablet_bootstrap.cc:492] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b: Bootstrap starting.
I20260619 20:36:39.235944 30430 tablet_bootstrap.cc:654] T 3b7a94de04be440795495e93183ad64e P 7c2c664c3b4047e4ac4382641e5c2bde: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:39.236366 30431 tablet_bootstrap.cc:654] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:39.236541 30430 log.cc:824] T 3b7a94de04be440795495e93183ad64e P 7c2c664c3b4047e4ac4382641e5c2bde: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:39.236471 30217 tablet_service.cc:1467] Processing CreateTablet for tablet 3b7a94de04be440795495e93183ad64e (DEFAULT_TABLE table=test-table [id=91afc70895dc4643851a29f188d70e5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:39.236812 30217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b7a94de04be440795495e93183ad64e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:39.236948 30431 log.cc:824] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:39.237398 30430 tablet_bootstrap.cc:492] T 3b7a94de04be440795495e93183ad64e P 7c2c664c3b4047e4ac4382641e5c2bde: No bootstrap required, opened a new log
I20260619 20:36:39.237469 30430 ts_tablet_manager.cc:1397] T 3b7a94de04be440795495e93183ad64e P 7c2c664c3b4047e4ac4382641e5c2bde: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:36:39.237723 30431 tablet_bootstrap.cc:492] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b: No bootstrap required, opened a new log
I20260619 20:36:39.237792 30431 ts_tablet_manager.cc:1397] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 20:36:39.238940 30430 raft_consensus.cc:348] T 3b7a94de04be440795495e93183ad64e P 7c2c664c3b4047e4ac4382641e5c2bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2c664c3b4047e4ac4382641e5c2bde" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 46793 } } peers { permanent_uuid: "e29d76830c1d4ba385928b92fa6e4c75" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37335 } } peers { permanent_uuid: "fe0fb72cca7e47828934d28edb1df76b" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36413 } }
I20260619 20:36:39.239091 30430 raft_consensus.cc:374] T 3b7a94de04be440795495e93183ad64e P 7c2c664c3b4047e4ac4382641e5c2bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:39.239133 30430 raft_consensus.cc:729] T 3b7a94de04be440795495e93183ad64e P 7c2c664c3b4047e4ac4382641e5c2bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c2c664c3b4047e4ac4382641e5c2bde, State: Initialized, Role: FOLLOWER
I20260619 20:36:39.239264 30430 consensus_queue.cc:260] T 3b7a94de04be440795495e93183ad64e P 7c2c664c3b4047e4ac4382641e5c2bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2c664c3b4047e4ac4382641e5c2bde" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 46793 } } peers { permanent_uuid: "e29d76830c1d4ba385928b92fa6e4c75" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37335 } } peers { permanent_uuid: "fe0fb72cca7e47828934d28edb1df76b" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36413 } }
I20260619 20:36:39.239539 30430 ts_tablet_manager.cc:1428] T 3b7a94de04be440795495e93183ad64e P 7c2c664c3b4047e4ac4382641e5c2bde: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:36:39.239652 30150 heartbeater.cc:503] Master 127.25.62.190:32907 was elected leader, sending a full tablet report...
I20260619 20:36:39.239799 30431 raft_consensus.cc:348] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2c664c3b4047e4ac4382641e5c2bde" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 46793 } } peers { permanent_uuid: "e29d76830c1d4ba385928b92fa6e4c75" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37335 } } peers { permanent_uuid: "fe0fb72cca7e47828934d28edb1df76b" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36413 } }
I20260619 20:36:39.239954 30431 raft_consensus.cc:374] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:39.239996 30431 raft_consensus.cc:729] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe0fb72cca7e47828934d28edb1df76b, State: Initialized, Role: FOLLOWER
I20260619 20:36:39.240119 30431 consensus_queue.cc:260] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2c664c3b4047e4ac4382641e5c2bde" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 46793 } } peers { permanent_uuid: "e29d76830c1d4ba385928b92fa6e4c75" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37335 } } peers { permanent_uuid: "fe0fb72cca7e47828934d28edb1df76b" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36413 } }
I20260619 20:36:39.240293 30435 tablet_bootstrap.cc:492] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75: Bootstrap starting.
I20260619 20:36:39.240372 30431 ts_tablet_manager.cc:1428] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 20:36:39.240386 30416 heartbeater.cc:503] Master 127.25.62.190:32907 was elected leader, sending a full tablet report...
I20260619 20:36:39.240949 30435 tablet_bootstrap.cc:654] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:39.241520 30435 log.cc:824] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:39.242141 30435 tablet_bootstrap.cc:492] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75: No bootstrap required, opened a new log
I20260619 20:36:39.242218 30435 ts_tablet_manager.cc:1397] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:36:39.243472 30435 raft_consensus.cc:348] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2c664c3b4047e4ac4382641e5c2bde" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 46793 } } peers { permanent_uuid: "e29d76830c1d4ba385928b92fa6e4c75" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37335 } } peers { permanent_uuid: "fe0fb72cca7e47828934d28edb1df76b" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36413 } }
I20260619 20:36:39.243660 30435 raft_consensus.cc:374] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:39.243708 30435 raft_consensus.cc:729] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e29d76830c1d4ba385928b92fa6e4c75, State: Initialized, Role: FOLLOWER
I20260619 20:36:39.243852 30435 consensus_queue.cc:260] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2c664c3b4047e4ac4382641e5c2bde" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 46793 } } peers { permanent_uuid: "e29d76830c1d4ba385928b92fa6e4c75" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37335 } } peers { permanent_uuid: "fe0fb72cca7e47828934d28edb1df76b" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36413 } }
I20260619 20:36:39.244061 30435 ts_tablet_manager.cc:1428] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:36:39.244108 30283 heartbeater.cc:503] Master 127.25.62.190:32907 was elected leader, sending a full tablet report...
W20260619 20:36:39.289314 30284 tablet.cc:2369] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:36:39.384833 30151 tablet.cc:2369] T 3b7a94de04be440795495e93183ad64e P 7c2c664c3b4047e4ac4382641e5c2bde: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:36:39.443982 30417 tablet.cc:2369] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:36:39.526154 30436 raft_consensus.cc:482] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:36:39.526276 30436 raft_consensus.cc:504] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2c664c3b4047e4ac4382641e5c2bde" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 46793 } } peers { permanent_uuid: "e29d76830c1d4ba385928b92fa6e4c75" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37335 } } peers { permanent_uuid: "fe0fb72cca7e47828934d28edb1df76b" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36413 } }
I20260619 20:36:39.526587 30436 leader_election.cc:288] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c2c664c3b4047e4ac4382641e5c2bde (127.25.62.129:46793), e29d76830c1d4ba385928b92fa6e4c75 (127.25.62.130:37335)
I20260619 20:36:39.530684 30425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.526932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:36515 (local address 127.25.62.129:46793)
0619 20:36:39.526983 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:39.526986 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:39.526997 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:36:39.527029 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:39.527030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:39.527042 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:39.527153 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:39.527320 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.528127 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.529719 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.530282 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.530353 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:39.530361 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:39.530576 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 20:36:39.530614 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 20:36:39.530684 30428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.527010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:48413 (local address 127.25.62.130:37335)
0619 20:36:39.527065 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:39.527067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:39.527169 (+ 102us) server_negotiation.cc:407] Connection header received
0619 20:36:39.527202 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:39.527203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:39.527212 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:39.527271 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:39.527429 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.528144 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.529719 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.530268 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.530352 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:39.530361 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:39.530576 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 20:36:39.530614 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 20:36:39.530709 30418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.526792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:46793 (local address 127.25.62.131:36515)
0619 20:36:39.526853 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:39.526866 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:39.526931 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:39.527144 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:39.527145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:39.527147 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:39.527233 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:39.527234 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.528238 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.528239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:39.529619 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:39.529622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.530385 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.530386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:39.530486 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:39.530488 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:36:39.530557 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 20:36:39.530609 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":94}
I20260619 20:36:39.530714 30439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.526910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:37335 (local address 127.25.62.131:48413)
0619 20:36:39.527064 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:39.527073 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:39.527149 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:39.527295 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:39.527295 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:39.527297 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:39.527360 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:39.527361 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.528238 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.528239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:39.529619 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:39.529622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.530385 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.530386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:39.530483 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:39.530487 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:39.530557 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 20:36:39.530609 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":39,"thread_start_us":111,"threads_started":1}
I20260619 20:36:39.531098 30104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b7a94de04be440795495e93183ad64e" candidate_uuid: "fe0fb72cca7e47828934d28edb1df76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c2c664c3b4047e4ac4382641e5c2bde" is_pre_election: true
I20260619 20:36:39.531266 30104 raft_consensus.cc:2454] T 3b7a94de04be440795495e93183ad64e P 7c2c664c3b4047e4ac4382641e5c2bde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe0fb72cca7e47828934d28edb1df76b in term 0.
I20260619 20:36:39.531212 30237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b7a94de04be440795495e93183ad64e" candidate_uuid: "fe0fb72cca7e47828934d28edb1df76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e29d76830c1d4ba385928b92fa6e4c75" is_pre_election: true
I20260619 20:36:39.531394 30237 raft_consensus.cc:2454] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe0fb72cca7e47828934d28edb1df76b in term 0.
I20260619 20:36:39.531509 30304 leader_election.cc:302] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c2c664c3b4047e4ac4382641e5c2bde, fe0fb72cca7e47828934d28edb1df76b; no voters:
I20260619 20:36:39.531673 30436 raft_consensus.cc:2781] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:36:39.531750 30436 raft_consensus.cc:482] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:36:39.531769 30436 raft_consensus.cc:3037] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:39.532480 30436 raft_consensus.cc:504] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2c664c3b4047e4ac4382641e5c2bde" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 46793 } } peers { permanent_uuid: "e29d76830c1d4ba385928b92fa6e4c75" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37335 } } peers { permanent_uuid: "fe0fb72cca7e47828934d28edb1df76b" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36413 } }
I20260619 20:36:39.532606 30436 leader_election.cc:288] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [CANDIDATE]: Term 1 election: Requested vote from peers 7c2c664c3b4047e4ac4382641e5c2bde (127.25.62.129:46793), e29d76830c1d4ba385928b92fa6e4c75 (127.25.62.130:37335)
I20260619 20:36:39.532804 30237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b7a94de04be440795495e93183ad64e" candidate_uuid: "fe0fb72cca7e47828934d28edb1df76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e29d76830c1d4ba385928b92fa6e4c75"
I20260619 20:36:39.532788 30104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b7a94de04be440795495e93183ad64e" candidate_uuid: "fe0fb72cca7e47828934d28edb1df76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c2c664c3b4047e4ac4382641e5c2bde"
I20260619 20:36:39.532881 30104 raft_consensus.cc:3037] T 3b7a94de04be440795495e93183ad64e P 7c2c664c3b4047e4ac4382641e5c2bde [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:39.532884 30237 raft_consensus.cc:3037] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:39.533619 30237 raft_consensus.cc:2454] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe0fb72cca7e47828934d28edb1df76b in term 1.
I20260619 20:36:39.533622 30104 raft_consensus.cc:2454] T 3b7a94de04be440795495e93183ad64e P 7c2c664c3b4047e4ac4382641e5c2bde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe0fb72cca7e47828934d28edb1df76b in term 1.
I20260619 20:36:39.533808 30304 leader_election.cc:302] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e29d76830c1d4ba385928b92fa6e4c75, fe0fb72cca7e47828934d28edb1df76b; no voters:
I20260619 20:36:39.533919 30436 raft_consensus.cc:2781] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:39.534094 30436 raft_consensus.cc:686] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [term 1 LEADER]: Becoming Leader. State: Replica: fe0fb72cca7e47828934d28edb1df76b, State: Running, Role: LEADER
I20260619 20:36:39.534199 30436 consensus_queue.cc:237] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2c664c3b4047e4ac4382641e5c2bde" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 46793 } } peers { permanent_uuid: "e29d76830c1d4ba385928b92fa6e4c75" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37335 } } peers { permanent_uuid: "fe0fb72cca7e47828934d28edb1df76b" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36413 } }
I20260619 20:36:39.534961 29960 catalog_manager.cc:5495] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b reported cstate change: term changed from 0 to 1, leader changed from <none> to fe0fb72cca7e47828934d28edb1df76b (127.25.62.131). New cstate: current_term: 1 leader_uuid: "fe0fb72cca7e47828934d28edb1df76b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2c664c3b4047e4ac4382641e5c2bde" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 46793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e29d76830c1d4ba385928b92fa6e4c75" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe0fb72cca7e47828934d28edb1df76b" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36413 } health_report { overall_health: HEALTHY } } }
I20260619 20:36:39.584141 30429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.581683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49932 (local address 127.25.62.131:36413)
0619 20:36:39.581756 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:39.581758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:39.581768 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:36:39.581817 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:39.581818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:39.581836 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:39.581915 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:39.582093 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.582568 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.583320 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:39.583537 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:39.583599 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:39.583754 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:39.583858 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:39.583936 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:39.583970 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 20:36:39.583989 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 20:36:39.586040 30237 raft_consensus.cc:1261] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75 [term 1 FOLLOWER]: Refusing update from remote peer fe0fb72cca7e47828934d28edb1df76b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:39.586138 30104 raft_consensus.cc:1261] T 3b7a94de04be440795495e93183ad64e P 7c2c664c3b4047e4ac4382641e5c2bde [term 1 FOLLOWER]: Refusing update from remote peer fe0fb72cca7e47828934d28edb1df76b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:39.586434 30436 consensus_queue.cc:1035] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e29d76830c1d4ba385928b92fa6e4c75" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:39.586572 30440 consensus_queue.cc:1035] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c2c664c3b4047e4ac4382641e5c2bde" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 46793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:39.588944 30443 mvcc.cc:205] Tried to move back new op lower bound from 7298668132701302784 to 7298668132492713984. Current Snapshot: MvccSnapshot[applied={T|T < 7298668132701302784}]
I20260619 20:36:39.589078 30444 mvcc.cc:205] Tried to move back new op lower bound from 7298668132701302784 to 7298668132492713984. Current Snapshot: MvccSnapshot[applied={T|T < 7298668132701302784}]
I20260619 20:36:39.589264 30445 mvcc.cc:205] Tried to move back new op lower bound from 7298668132701302784 to 7298668132492713984. Current Snapshot: MvccSnapshot[applied={T|T < 7298668132701302784}]
I20260619 20:36:39.591403 29959 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:39.591492 29959 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:39.592275 29959 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4eb4edcd52e8487c849f1cdddd312e71: Sending DeleteTablet for 3 replicas of tablet 3b7a94de04be440795495e93183ad64e
I20260619 20:36:39.592553 30084 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b7a94de04be440795495e93183ad64e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:39 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:54910
I20260619 20:36:39.592644 30350 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b7a94de04be440795495e93183ad64e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:39 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:49924
I20260619 20:36:39.592649 30217 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b7a94de04be440795495e93183ad64e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:39 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:60774
I20260619 20:36:39.592763 30453 tablet_replica.cc:330] T 3b7a94de04be440795495e93183ad64e P 7c2c664c3b4047e4ac4382641e5c2bde: stopping tablet replica
I20260619 20:36:39.592850 30453 raft_consensus.cc:2229] T 3b7a94de04be440795495e93183ad64e P 7c2c664c3b4047e4ac4382641e5c2bde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:39.592959 30453 raft_consensus.cc:2258] T 3b7a94de04be440795495e93183ad64e P 7c2c664c3b4047e4ac4382641e5c2bde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:39.593106 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 30022
I20260619 20:36:39.593673 30454 tablet_replica.cc:330] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75: stopping tablet replica
I20260619 20:36:39.593768 30454 raft_consensus.cc:2229] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:39.593859 30454 raft_consensus.cc:2258] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:39.593989 30455 tablet_replica.cc:330] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b: stopping tablet replica
I20260619 20:36:39.594066 30455 raft_consensus.cc:2229] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:36:39.594172 30455 raft_consensus.cc:2258] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:39.594707 30455 ts_tablet_manager.cc:1905] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:39.596024 30454 ts_tablet_manager.cc:1905] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:39.596248 30455 ts_tablet_manager.cc:1918] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:36:39.596364 30455 log.cc:1196] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/3b7a94de04be440795495e93183ad64e
I20260619 20:36:39.596683 30455 ts_tablet_manager.cc:1939] T 3b7a94de04be440795495e93183ad64e P fe0fb72cca7e47828934d28edb1df76b: Deleting consensus metadata
I20260619 20:36:39.597155 29945 catalog_manager.cc:4841] TS fe0fb72cca7e47828934d28edb1df76b (127.25.62.131:36413): tablet 3b7a94de04be440795495e93183ad64e (table test-table [id=91afc70895dc4643851a29f188d70e5a]) successfully deleted
I20260619 20:36:39.597615 30454 ts_tablet_manager.cc:1918] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:36:39.597702 30454 log.cc:1196] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/3b7a94de04be440795495e93183ad64e
I20260619 20:36:39.597976 30454 ts_tablet_manager.cc:1939] T 3b7a94de04be440795495e93183ad64e P e29d76830c1d4ba385928b92fa6e4c75: Deleting consensus metadata
I20260619 20:36:39.598489 29945 catalog_manager.cc:4841] TS e29d76830c1d4ba385928b92fa6e4c75 (127.25.62.130:37335): tablet 3b7a94de04be440795495e93183ad64e (table test-table [id=91afc70895dc4643851a29f188d70e5a]) successfully deleted
W20260619 20:36:39.599376 29945 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.62.129:46793 (error 108)
I20260619 20:36:39.599628 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 30155
I20260619 20:36:39.600194 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.599738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:46793 (local address 127.0.0.1:54922)
0619 20:36:39.600125 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:39.600158 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.129:46793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370}
W20260619 20:36:39.601963 29945 catalog_manager.cc:4584] TS 7c2c664c3b4047e4ac4382641e5c2bde (127.25.62.129:46793): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3b7a94de04be440795495e93183ad64e: Network error: Client connection negotiation failed: client connection to 127.25.62.129:46793: connect: Connection refused (error 111)
I20260619 20:36:39.604976 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 30288
I20260619 20:36:39.610685 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 29929
2026-06-19T20:36:39Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2191 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T20:36:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:39Z Disabled control of system clock
I20260619 20:36:39.637236 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:44145
--webserver_interface=127.25.62.190
--webserver_port=0
--builtin_ntp_servers=127.25.62.148:38519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:44145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 20:36:39.715529 30460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:39.715734 30460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:39.715754 30460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:39.716846 30460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:39.716881 30460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:39.716893 30460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:39.716907 30460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:39.716918 30460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:39.718200 30460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:38519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:44145
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:44145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:39.718386 30460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:39.718611 30460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:39.723322 30465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:39.723469 30460 server_base.cc:1034] running on GCE node
W20260619 20:36:39.723308 30468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:39.723331 30466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:39.723872 30460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:39.724169 30460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:39.725311 30460 hybrid_clock.cc:648] HybridClock initialized: now 1781901399725281 us; error 49 us; skew 500 ppm
I20260619 20:36:39.726536 30460 webserver.cc:460] Webserver started at http://127.25.62.190:36921/ using document root <none> and password file <none>
I20260619 20:36:39.726725 30460 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:39.726764 30460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:39.726854 30460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:39.727895 30460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "77f74961d1214000a28852eee810e8d4"
format_stamp: "Formatted at 2026-06-19 20:36:39 on dist-test-slave-0t40"
server_key: "f0a158f2db44eaf8b225340f5ebfd546"
server_key_iv: "71246dea086c35cec72bf6b63f104128"
server_key_version: "encryptionkey@0"
I20260619 20:36:39.728202 30460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "77f74961d1214000a28852eee810e8d4"
format_stamp: "Formatted at 2026-06-19 20:36:39 on dist-test-slave-0t40"
server_key: "f0a158f2db44eaf8b225340f5ebfd546"
server_key_iv: "71246dea086c35cec72bf6b63f104128"
server_key_version: "encryptionkey@0"
I20260619 20:36:39.729442 30460 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:36:39.730137 30474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:39.730319 30460 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:36:39.730405 30460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "77f74961d1214000a28852eee810e8d4"
format_stamp: "Formatted at 2026-06-19 20:36:39 on dist-test-slave-0t40"
server_key: "f0a158f2db44eaf8b225340f5ebfd546"
server_key_iv: "71246dea086c35cec72bf6b63f104128"
server_key_version: "encryptionkey@0"
I20260619 20:36:39.730486 30460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:39.733788 30460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:39.734054 30460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:39.734190 30460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:39.738076 30460 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:44145
I20260619 20:36:39.738106 30526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:44145 every 8 connection(s)
I20260619 20:36:39.738425 30460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:39.739029 30527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:39.740918 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 30460
I20260619 20:36:39.741024 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:36:39.741220 25850 external_mini_cluster.cc:1428] Setting key da8b72d8f16ec0d2980f1e257495ff6c
I20260619 20:36:39.742030 30527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4: Bootstrap starting.
I20260619 20:36:39.742650 30527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:39.743003 30527 log.cc:824] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:39.743700 30530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.741721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53536 (local address 127.25.62.190:44145)
0619 20:36:39.741900 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:39.741906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:39.741930 (+ 24us) server_negotiation.cc:407] Connection header received
0619 20:36:39.742387 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:39.742394 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:39.742434 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:39.742550 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:39.743211 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:39.743215 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:39.743216 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:39.743221 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:39.743265 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:39.743278 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:39.743339 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:39.743461 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 20:36:39.743523 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":73,"threads_started":1}
I20260619 20:36:39.743767 30527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4: No bootstrap required, opened a new log
I20260619 20:36:39.745297 30527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77f74961d1214000a28852eee810e8d4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44145 } }
I20260619 20:36:39.745424 30527 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:39.745445 30527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77f74961d1214000a28852eee810e8d4, State: Initialized, Role: FOLLOWER
I20260619 20:36:39.745587 30527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77f74961d1214000a28852eee810e8d4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44145 } }
I20260619 20:36:39.745661 30527 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:39.745703 30527 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:39.745751 30527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:39.746507 30527 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77f74961d1214000a28852eee810e8d4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44145 } }
I20260619 20:36:39.746594 30527 leader_election.cc:302] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77f74961d1214000a28852eee810e8d4; no voters:
I20260619 20:36:39.746771 30527 leader_election.cc:288] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:39.746819 30532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:39.747030 30532 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4 [term 1 LEADER]: Becoming Leader. State: Replica: 77f74961d1214000a28852eee810e8d4, State: Running, Role: LEADER
I20260619 20:36:39.747066 30527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:39.747160 30532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77f74961d1214000a28852eee810e8d4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44145 } }
I20260619 20:36:39.747519 30534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77f74961d1214000a28852eee810e8d4. Latest consensus state: current_term: 1 leader_uuid: "77f74961d1214000a28852eee810e8d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77f74961d1214000a28852eee810e8d4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44145 } } }
I20260619 20:36:39.747519 30533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77f74961d1214000a28852eee810e8d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77f74961d1214000a28852eee810e8d4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44145 } } }
I20260619 20:36:39.747658 30534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:39.747663 30533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:39.747864 30541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:39.748329 30541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:39.749889 30541 catalog_manager.cc:1347] Generated new cluster ID: 8e2db6e2201442e68d3f2dae79a51724
I20260619 20:36:39.749956 30541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:39.758870 30541 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:39.759048 30541 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:39.766163 30541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4: Generated new TSK 0
I20260619 20:36:39.766374 30541 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:36:39.769685 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:44145
--builtin_ntp_servers=127.25.62.148:38519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 20:36:39.852159 30551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:39.852413 30551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:39.852455 30551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:39.853695 30551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:39.853761 30551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:39.853807 30551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:36:39.855201 30551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:38519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:44145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:39.855471 30551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:39.855768 30551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:39.861515 30556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:39.861505 30557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:39.861744 30551 server_base.cc:1034] running on GCE node
W20260619 20:36:39.861501 30559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:39.861986 30551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:39.862227 30551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:39.863397 30551 hybrid_clock.cc:648] HybridClock initialized: now 1781901399863381 us; error 32 us; skew 500 ppm
I20260619 20:36:39.864820 30551 webserver.cc:460] Webserver started at http://127.25.62.129:33557/ using document root <none> and password file <none>
I20260619 20:36:39.865172 30551 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:39.865249 30551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:39.865371 30551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:39.866827 30551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "e1e188af41ad458dbd316c405d8c0ecc"
format_stamp: "Formatted at 2026-06-19 20:36:39 on dist-test-slave-0t40"
server_key: "f25551d5868962998e296fe41317cb66"
server_key_iv: "c3936d3fca5ee0a378a3b27ff69defc8"
server_key_version: "encryptionkey@0"
I20260619 20:36:39.867285 30551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "e1e188af41ad458dbd316c405d8c0ecc"
format_stamp: "Formatted at 2026-06-19 20:36:39 on dist-test-slave-0t40"
server_key: "f25551d5868962998e296fe41317cb66"
server_key_iv: "c3936d3fca5ee0a378a3b27ff69defc8"
server_key_version: "encryptionkey@0"
I20260619 20:36:39.868932 30551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260619 20:36:39.869841 30565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:39.870052 30551 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:39.870136 30551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "e1e188af41ad458dbd316c405d8c0ecc"
format_stamp: "Formatted at 2026-06-19 20:36:39 on dist-test-slave-0t40"
server_key: "f25551d5868962998e296fe41317cb66"
server_key_iv: "c3936d3fca5ee0a378a3b27ff69defc8"
server_key_version: "encryptionkey@0"
I20260619 20:36:39.870198 30551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:39.874759 30551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:39.875059 30551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:39.875197 30551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:39.875409 30551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:39.875710 30551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:39.875748 30551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:39.875773 30551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:39.875789 30551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:39.882117 30551 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:40131
I20260619 20:36:39.882175 30678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:40131 every 8 connection(s)
I20260619 20:36:39.882485 30551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:36:39.884590 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 30551
I20260619 20:36:39.884661 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:36:39.884852 25850 external_mini_cluster.cc:1428] Setting key d87f7bffaca348b3a40345ce393de14c
I20260619 20:36:39.885838 30530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.883261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:36979 (local address 127.25.62.190:44145)
0619 20:36:39.883334 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:39.883336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:39.883569 (+ 233us) server_negotiation.cc:407] Connection header received
0619 20:36:39.884010 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:39.884011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:39.884015 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:39.884067 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:39.885593 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:39.885594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:39.885595 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:39.885599 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:39.885615 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:39.885619 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:39.885676 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:39.885775 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 20:36:39.885787 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 20:36:39.885987 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:44145
--builtin_ntp_servers=127.25.62.148:38519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 20:36:39.886102 30681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.883225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:44145 (local address 127.25.62.129:36979)
0619 20:36:39.883435 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:39.883475 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:39.883913 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:39.884118 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:39.884121 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:39.884129 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:39.885391 (+ 1262us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:36:39.885392 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:39.885421 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:36:39.885450 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:39.885685 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:39.885691 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:39.885694 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:36:39.885784 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 20:36:39.885850 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":89,"threads_started":1}
I20260619 20:36:39.887578 30679 heartbeater.cc:347] Connected to a master server at 127.25.62.190:44145
I20260619 20:36:39.887677 30679 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:39.887858 30679 heartbeater.cc:511] Master 127.25.62.190:44145 requested a full tablet report, sending...
I20260619 20:36:39.888391 30491 ts_manager.cc:194] Registered new tserver with Master: e1e188af41ad458dbd316c405d8c0ecc (127.25.62.129:40131)
W20260619 20:36:39.968007 30682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:39.968216 30682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:39.968241 30682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:39.969367 30682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:39.969405 30682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:39.969432 30682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:36:39.970701 30682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:38519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:44145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:39.970912 30682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:39.971154 30682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:39.976447 30688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:39.976485 30690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:39.976447 30687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:39.976806 30682 server_base.cc:1034] running on GCE node
I20260619 20:36:39.977113 30682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:39.977353 30682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:39.978530 30682 hybrid_clock.cc:648] HybridClock initialized: now 1781901399978512 us; error 32 us; skew 500 ppm
I20260619 20:36:39.980059 30682 webserver.cc:460] Webserver started at http://127.25.62.130:36433/ using document root <none> and password file <none>
I20260619 20:36:39.980329 30682 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:39.980376 30682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:39.980545 30682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:39.981698 30682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "5d010684da6f4ab09b7eb9f473926a12"
format_stamp: "Formatted at 2026-06-19 20:36:39 on dist-test-slave-0t40"
server_key: "78a4d5bbaa5a78572c8d9e203bb6af6d"
server_key_iv: "a9e2084ee1c9b6e82fe0bc3f951c2ec8"
server_key_version: "encryptionkey@0"
I20260619 20:36:39.982055 30682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "5d010684da6f4ab09b7eb9f473926a12"
format_stamp: "Formatted at 2026-06-19 20:36:39 on dist-test-slave-0t40"
server_key: "78a4d5bbaa5a78572c8d9e203bb6af6d"
server_key_iv: "a9e2084ee1c9b6e82fe0bc3f951c2ec8"
server_key_version: "encryptionkey@0"
I20260619 20:36:39.983314 30682 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 20:36:39.984025 30696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:39.984216 30682 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:39.984310 30682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "5d010684da6f4ab09b7eb9f473926a12"
format_stamp: "Formatted at 2026-06-19 20:36:39 on dist-test-slave-0t40"
server_key: "78a4d5bbaa5a78572c8d9e203bb6af6d"
server_key_iv: "a9e2084ee1c9b6e82fe0bc3f951c2ec8"
server_key_version: "encryptionkey@0"
I20260619 20:36:39.984385 30682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:39.987756 30682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:39.988016 30682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:39.988152 30682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:39.988349 30682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:39.988646 30682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:39.988706 30682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:39.988745 30682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:39.988772 30682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:39.994733 30682 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:40865
I20260619 20:36:39.994771 30809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:40865 every 8 connection(s)
I20260619 20:36:39.995106 30682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:36:39.998185 30530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.995830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:37143 (local address 127.25.62.190:44145)
0619 20:36:39.995899 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:39.995901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:39.996161 (+ 260us) server_negotiation.cc:407] Connection header received
0619 20:36:39.996645 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:39.996646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:39.996653 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:39.996700 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:39.997911 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:39.997913 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:39.997913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:39.997919 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:39.997943 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:39.997947 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:39.998004 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:39.998097 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 20:36:39.998113 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 20:36:39.998399 30812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:39.995834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:44145 (local address 127.25.62.130:37143)
0619 20:36:39.996043 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:39.996079 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:39.996547 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:39.996769 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:39.996773 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:39.996780 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:39.997760 (+ 980us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:36:39.997761 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:39.997788 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:36:39.997816 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:39.998013 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:39.998019 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:39.998022 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:36:39.998107 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 20:36:39.998167 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":108,"threads_started":1}
I20260619 20:36:39.998874 30810 heartbeater.cc:347] Connected to a master server at 127.25.62.190:44145
I20260619 20:36:39.998962 30810 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:39.999073 30810 heartbeater.cc:511] Master 127.25.62.190:44145 requested a full tablet report, sending...
I20260619 20:36:39.999377 30491 ts_manager.cc:194] Registered new tserver with Master: 5d010684da6f4ab09b7eb9f473926a12 (127.25.62.130:40865)
I20260619 20:36:40.001204 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 30682
I20260619 20:36:40.001281 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:36:40.001464 25850 external_mini_cluster.cc:1428] Setting key 528eff918070527d06a7b40a119c8547
I20260619 20:36:40.002331 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:44145
--builtin_ntp_servers=127.25.62.148:38519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 20:36:40.081511 30813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:40.081693 30813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:40.081714 30813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:40.082795 30813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:40.082828 30813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:40.082852 30813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:36:40.084082 30813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:38519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:44145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:40.084255 30813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:40.084481 30813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:40.089403 30818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:40.089416 30821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:40.089526 30819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:40.089531 30813 server_base.cc:1034] running on GCE node
I20260619 20:36:40.089898 30813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:40.090098 30813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:40.091235 30813 hybrid_clock.cc:648] HybridClock initialized: now 1781901400091221 us; error 27 us; skew 500 ppm
I20260619 20:36:40.092439 30813 webserver.cc:460] Webserver started at http://127.25.62.131:40007/ using document root <none> and password file <none>
I20260619 20:36:40.092630 30813 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:40.092671 30813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:40.092757 30813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:40.093796 30813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "508d22e5059c4c5f9f5cdbb8a157bcef"
format_stamp: "Formatted at 2026-06-19 20:36:40 on dist-test-slave-0t40"
server_key: "874b0a14354de3da57a892148edb4dbd"
server_key_iv: "143ad40e3a587ee3c2a8c290f8e5e43e"
server_key_version: "encryptionkey@0"
I20260619 20:36:40.094087 30813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "508d22e5059c4c5f9f5cdbb8a157bcef"
format_stamp: "Formatted at 2026-06-19 20:36:40 on dist-test-slave-0t40"
server_key: "874b0a14354de3da57a892148edb4dbd"
server_key_iv: "143ad40e3a587ee3c2a8c290f8e5e43e"
server_key_version: "encryptionkey@0"
I20260619 20:36:40.095305 30813 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:36:40.095963 30827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:40.096143 30813 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:36:40.096225 30813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "508d22e5059c4c5f9f5cdbb8a157bcef"
format_stamp: "Formatted at 2026-06-19 20:36:40 on dist-test-slave-0t40"
server_key: "874b0a14354de3da57a892148edb4dbd"
server_key_iv: "143ad40e3a587ee3c2a8c290f8e5e43e"
server_key_version: "encryptionkey@0"
I20260619 20:36:40.096278 30813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:40.099426 30813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:40.099614 30813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:40.099702 30813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:40.099836 30813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:40.100077 30813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:40.100106 30813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:40.100124 30813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:40.100136 30813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:40.105808 30813 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:33079
I20260619 20:36:40.105885 30940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:33079 every 8 connection(s)
I20260619 20:36:40.106191 30813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:36:40.106719 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 30813
I20260619 20:36:40.106798 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:36:40.106971 25850 external_mini_cluster.cc:1428] Setting key ad61203e1f67c9f07d82b83ea4f16797
I20260619 20:36:40.109091 30530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.106882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:60133 (local address 127.25.62.190:44145)
0619 20:36:40.106950 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:40.106952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:40.107197 (+ 245us) server_negotiation.cc:407] Connection header received
0619 20:36:40.107664 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:40.107666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:40.107672 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:40.107719 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:40.108831 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:40.108832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:40.108833 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:40.108838 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:40.108855 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:40.108859 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:40.108912 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:40.108988 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 20:36:40.109003 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 20:36:40.109282 30943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.106866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:44145 (local address 127.25.62.131:60133)
0619 20:36:40.107050 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:40.107091 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:40.107569 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:40.107783 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:40.107787 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:40.107796 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:40.108668 (+ 872us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:36:40.108668 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:40.108695 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:36:40.108723 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:40.108920 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:40.108925 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:40.108928 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:36:40.109000 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 20:36:40.109076 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
I20260619 20:36:40.109678 30941 heartbeater.cc:347] Connected to a master server at 127.25.62.190:44145
I20260619 20:36:40.109754 30941 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:40.109858 30941 heartbeater.cc:511] Master 127.25.62.190:44145 requested a full tablet report, sending...
I20260619 20:36:40.110158 30491 ts_manager.cc:194] Registered new tserver with Master: 508d22e5059c4c5f9f5cdbb8a157bcef (127.25.62.131:33079)
I20260619 20:36:40.118532 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:36:40.120957 30530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.119819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53544 (local address 127.25.62.190:44145)
0619 20:36:40.119898 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:40.119900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:40.119985 (+ 85us) server_negotiation.cc:407] Connection header received
0619 20:36:40.120050 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:40.120051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:40.120057 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:40.120113 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:40.120751 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:40.120752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:40.120752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:40.120757 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:40.120771 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:40.120775 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:40.120820 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:40.120874 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 20:36:40.120887 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 20:36:40.122599 30491 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:36:40.124040 30491 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:36:40.127643 30953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.126251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:33079 (local address 127.0.0.1:37990)
0619 20:36:40.126433 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:40.126442 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:40.126488 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:40.126736 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:40.126742 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:40.126750 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:40.127211 (+ 461us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:36:40.127212 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:40.127227 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:36:40.127250 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:40.127460 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:40.127467 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:40.127470 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:36:40.127539 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 20:36:40.127552 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":59,"threads_started":1}
I20260619 20:36:40.127825 30952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.126160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50094 (local address 127.25.62.129:40131)
0619 20:36:40.126601 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:40.126605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:40.126615 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:36:40.126639 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:40.126641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:40.126649 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:40.126717 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:40.127473 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:40.127479 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:40.127479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:40.127484 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:40.127513 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:40.127534 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:40.127589 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:40.127712 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 20:36:40.127733 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":83,"threads_started":1}
I20260619 20:36:40.127971 30949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.126070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:40131 (local address 127.0.0.1:50094)
0619 20:36:40.126230 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:40.126267 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:40.126323 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:40.126983 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:40.126984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:40.126986 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:40.127369 (+ 383us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:36:40.127370 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:40.127377 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:36:40.127389 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:40.127620 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:40.127621 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:40.127621 (+ 0us) client_negotiation.cc:769] Sending connection context
0619 20:36:40.127693 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 20:36:40.127827 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":50,"threads_started":1}
I20260619 20:36:40.128079 30954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.126434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37990 (local address 127.25.62.131:33079)
0619 20:36:40.126587 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:40.126591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:40.126601 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:36:40.126627 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:40.126629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:40.126636 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:40.126717 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:40.127315 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:40.127321 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:40.127322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:40.127326 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:40.127360 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:40.127382 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:40.127459 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:40.127814 (+ 355us) server_negotiation.cc:299] Negotiation successful
0619 20:36:40.127831 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260619 20:36:40.128569 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.126071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:40865 (local address 127.0.0.1:51516)
0619 20:36:40.126337 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:40.126349 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:40.126403 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:40.126891 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:40.126892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:40.126895 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:40.127974 (+ 1079us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:36:40.127975 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:40.127980 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:36:40.127991 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:40.128442 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:40.128443 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:40.128444 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 20:36:40.128502 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 20:36:40.128514 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":92,"threads_started":1}
I20260619 20:36:40.128607 30951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.126149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51516 (local address 127.25.62.130:40865)
0619 20:36:40.126329 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:40.126334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:40.126404 (+ 70us) server_negotiation.cc:407] Connection header received
0619 20:36:40.126473 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:40.126476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:40.126487 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:40.126560 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:40.128303 (+ 1743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:40.128312 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:40.128312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:40.128316 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:40.128348 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:40.128374 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:40.128436 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:40.128505 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 20:36:40.128522 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":69,"threads_started":1}
I20260619 20:36:40.128667 30613 tablet_service.cc:1467] Processing CreateTablet for tablet 1d1d1414813842b987bfa5b863c1542e (DEFAULT_TABLE table=test-table [id=f7a39ea1fc9c465a8cd4f70e3630965d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:40.128701 30875 tablet_service.cc:1467] Processing CreateTablet for tablet 1d1d1414813842b987bfa5b863c1542e (DEFAULT_TABLE table=test-table [id=f7a39ea1fc9c465a8cd4f70e3630965d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:40.129019 30613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d1d1414813842b987bfa5b863c1542e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:40.129027 30875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d1d1414813842b987bfa5b863c1542e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:40.131331 30955 tablet_bootstrap.cc:492] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc: Bootstrap starting.
I20260619 20:36:40.131383 30956 tablet_bootstrap.cc:492] T 1d1d1414813842b987bfa5b863c1542e P 508d22e5059c4c5f9f5cdbb8a157bcef: Bootstrap starting.
I20260619 20:36:40.131980 30956 tablet_bootstrap.cc:654] T 1d1d1414813842b987bfa5b863c1542e P 508d22e5059c4c5f9f5cdbb8a157bcef: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:40.132006 30955 tablet_bootstrap.cc:654] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:40.132288 30956 log.cc:824] T 1d1d1414813842b987bfa5b863c1542e P 508d22e5059c4c5f9f5cdbb8a157bcef: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:40.132301 30955 log.cc:824] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:40.132251 30743 tablet_service.cc:1467] Processing CreateTablet for tablet 1d1d1414813842b987bfa5b863c1542e (DEFAULT_TABLE table=test-table [id=f7a39ea1fc9c465a8cd4f70e3630965d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:40.132613 30743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d1d1414813842b987bfa5b863c1542e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:40.133004 30956 tablet_bootstrap.cc:492] T 1d1d1414813842b987bfa5b863c1542e P 508d22e5059c4c5f9f5cdbb8a157bcef: No bootstrap required, opened a new log
I20260619 20:36:40.133096 30956 ts_tablet_manager.cc:1397] T 1d1d1414813842b987bfa5b863c1542e P 508d22e5059c4c5f9f5cdbb8a157bcef: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:36:40.135082 30956 raft_consensus.cc:348] T 1d1d1414813842b987bfa5b863c1542e P 508d22e5059c4c5f9f5cdbb8a157bcef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d010684da6f4ab09b7eb9f473926a12" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 40865 } } peers { permanent_uuid: "e1e188af41ad458dbd316c405d8c0ecc" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40131 } } peers { permanent_uuid: "508d22e5059c4c5f9f5cdbb8a157bcef" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 33079 } }
I20260619 20:36:40.135265 30956 raft_consensus.cc:374] T 1d1d1414813842b987bfa5b863c1542e P 508d22e5059c4c5f9f5cdbb8a157bcef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:40.135263 30959 tablet_bootstrap.cc:492] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12: Bootstrap starting.
I20260619 20:36:40.135316 30956 raft_consensus.cc:729] T 1d1d1414813842b987bfa5b863c1542e P 508d22e5059c4c5f9f5cdbb8a157bcef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508d22e5059c4c5f9f5cdbb8a157bcef, State: Initialized, Role: FOLLOWER
I20260619 20:36:40.135433 30956 consensus_queue.cc:260] T 1d1d1414813842b987bfa5b863c1542e P 508d22e5059c4c5f9f5cdbb8a157bcef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d010684da6f4ab09b7eb9f473926a12" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 40865 } } peers { permanent_uuid: "e1e188af41ad458dbd316c405d8c0ecc" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40131 } } peers { permanent_uuid: "508d22e5059c4c5f9f5cdbb8a157bcef" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 33079 } }
I20260619 20:36:40.135707 30956 ts_tablet_manager.cc:1428] T 1d1d1414813842b987bfa5b863c1542e P 508d22e5059c4c5f9f5cdbb8a157bcef: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 20:36:40.135747 30941 heartbeater.cc:503] Master 127.25.62.190:44145 was elected leader, sending a full tablet report...
I20260619 20:36:40.135870 30959 tablet_bootstrap.cc:654] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:40.136126 30955 tablet_bootstrap.cc:492] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc: No bootstrap required, opened a new log
I20260619 20:36:40.136201 30955 ts_tablet_manager.cc:1397] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 20:36:40.136186 30959 log.cc:824] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:40.136925 30959 tablet_bootstrap.cc:492] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12: No bootstrap required, opened a new log
I20260619 20:36:40.136994 30959 ts_tablet_manager.cc:1397] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:36:40.138248 30955 raft_consensus.cc:348] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d010684da6f4ab09b7eb9f473926a12" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 40865 } } peers { permanent_uuid: "e1e188af41ad458dbd316c405d8c0ecc" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40131 } } peers { permanent_uuid: "508d22e5059c4c5f9f5cdbb8a157bcef" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 33079 } }
I20260619 20:36:40.138366 30955 raft_consensus.cc:374] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:40.138387 30955 raft_consensus.cc:729] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1e188af41ad458dbd316c405d8c0ecc, State: Initialized, Role: FOLLOWER
I20260619 20:36:40.138505 30955 consensus_queue.cc:260] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d010684da6f4ab09b7eb9f473926a12" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 40865 } } peers { permanent_uuid: "e1e188af41ad458dbd316c405d8c0ecc" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40131 } } peers { permanent_uuid: "508d22e5059c4c5f9f5cdbb8a157bcef" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 33079 } }
I20260619 20:36:40.138767 30955 ts_tablet_manager.cc:1428] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:36:40.138787 30679 heartbeater.cc:503] Master 127.25.62.190:44145 was elected leader, sending a full tablet report...
I20260619 20:36:40.138907 30959 raft_consensus.cc:348] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d010684da6f4ab09b7eb9f473926a12" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 40865 } } peers { permanent_uuid: "e1e188af41ad458dbd316c405d8c0ecc" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40131 } } peers { permanent_uuid: "508d22e5059c4c5f9f5cdbb8a157bcef" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 33079 } }
I20260619 20:36:40.139048 30959 raft_consensus.cc:374] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:40.139087 30959 raft_consensus.cc:729] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d010684da6f4ab09b7eb9f473926a12, State: Initialized, Role: FOLLOWER
I20260619 20:36:40.139196 30959 consensus_queue.cc:260] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d010684da6f4ab09b7eb9f473926a12" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 40865 } } peers { permanent_uuid: "e1e188af41ad458dbd316c405d8c0ecc" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40131 } } peers { permanent_uuid: "508d22e5059c4c5f9f5cdbb8a157bcef" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 33079 } }
I20260619 20:36:40.139453 30959 ts_tablet_manager.cc:1428] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:36:40.139592 30810 heartbeater.cc:503] Master 127.25.62.190:44145 was elected leader, sending a full tablet report...
W20260619 20:36:40.245628 30811 tablet.cc:2369] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:36:40.246048 30962 raft_consensus.cc:482] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:36:40.246160 30962 raft_consensus.cc:504] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d010684da6f4ab09b7eb9f473926a12" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 40865 } } peers { permanent_uuid: "e1e188af41ad458dbd316c405d8c0ecc" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40131 } } peers { permanent_uuid: "508d22e5059c4c5f9f5cdbb8a157bcef" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 33079 } }
I20260619 20:36:40.246575 30962 leader_election.cc:288] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d010684da6f4ab09b7eb9f473926a12 (127.25.62.130:40865), 508d22e5059c4c5f9f5cdbb8a157bcef (127.25.62.131:33079)
I20260619 20:36:40.247931 30681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.246737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:40865 (local address 127.25.62.129:35183)
0619 20:36:40.246819 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:40.246831 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:40.246867 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:40.247043 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:40.247044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:40.247047 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:40.247586 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:36:40.247586 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:40.247595 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:36:40.247609 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:40.247784 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:40.247784 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:40.247785 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 20:36:40.247850 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 20:36:40.247864 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 20:36:40.248126 30954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.246831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:54881 (local address 127.25.62.131:33079)
0619 20:36:40.246972 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:40.246975 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:40.247007 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:36:40.247074 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:40.247075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:40.247079 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:40.247136 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:40.247870 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:40.247871 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:40.247871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:40.247876 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:40.247893 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:40.247915 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:40.247977 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:40.248044 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 20:36:40.248060 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260619 20:36:40.248144 30964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.246750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:33079 (local address 127.25.62.129:54881)
0619 20:36:40.246962 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:40.246971 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:40.247013 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:40.247153 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:40.247154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:40.247155 (+ 1us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:40.247752 (+ 597us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:36:40.247752 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:40.247759 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:36:40.247772 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:40.247973 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:40.247974 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:40.247974 (+ 0us) client_negotiation.cc:769] Sending connection context
0619 20:36:40.248046 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 20:36:40.248062 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":21,"thread_start_us":106,"threads_started":1}
I20260619 20:36:40.248176 30951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.246806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:35183 (local address 127.25.62.130:40865)
0619 20:36:40.246894 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:40.246896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:40.246907 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:36:40.246933 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:40.246934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:40.246941 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:40.247023 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:40.247688 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:40.247690 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:40.247690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:40.247695 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:40.247711 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:40.247716 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:40.247773 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:40.248095 (+ 322us) server_negotiation.cc:299] Negotiation successful
0619 20:36:40.248109 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 20:36:40.248457 30764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d1d1414813842b987bfa5b863c1542e" candidate_uuid: "e1e188af41ad458dbd316c405d8c0ecc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d010684da6f4ab09b7eb9f473926a12" is_pre_election: true
I20260619 20:36:40.248627 30764 raft_consensus.cc:2454] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1e188af41ad458dbd316c405d8c0ecc in term 0.
I20260619 20:36:40.248606 30895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d1d1414813842b987bfa5b863c1542e" candidate_uuid: "e1e188af41ad458dbd316c405d8c0ecc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508d22e5059c4c5f9f5cdbb8a157bcef" is_pre_election: true
I20260619 20:36:40.248728 30895 raft_consensus.cc:2454] T 1d1d1414813842b987bfa5b863c1542e P 508d22e5059c4c5f9f5cdbb8a157bcef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1e188af41ad458dbd316c405d8c0ecc in term 0.
I20260619 20:36:40.248868 30567 leader_election.cc:302] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d010684da6f4ab09b7eb9f473926a12, e1e188af41ad458dbd316c405d8c0ecc; no voters:
I20260619 20:36:40.249010 30962 raft_consensus.cc:2781] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:36:40.249100 30962 raft_consensus.cc:482] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:36:40.249135 30962 raft_consensus.cc:3037] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:40.249786 30962 raft_consensus.cc:504] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d010684da6f4ab09b7eb9f473926a12" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 40865 } } peers { permanent_uuid: "e1e188af41ad458dbd316c405d8c0ecc" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40131 } } peers { permanent_uuid: "508d22e5059c4c5f9f5cdbb8a157bcef" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 33079 } }
I20260619 20:36:40.249919 30962 leader_election.cc:288] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc [CANDIDATE]: Term 1 election: Requested vote from peers 5d010684da6f4ab09b7eb9f473926a12 (127.25.62.130:40865), 508d22e5059c4c5f9f5cdbb8a157bcef (127.25.62.131:33079)
I20260619 20:36:40.250063 30895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d1d1414813842b987bfa5b863c1542e" candidate_uuid: "e1e188af41ad458dbd316c405d8c0ecc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508d22e5059c4c5f9f5cdbb8a157bcef"
I20260619 20:36:40.250144 30895 raft_consensus.cc:3037] T 1d1d1414813842b987bfa5b863c1542e P 508d22e5059c4c5f9f5cdbb8a157bcef [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:40.250871 30895 raft_consensus.cc:2454] T 1d1d1414813842b987bfa5b863c1542e P 508d22e5059c4c5f9f5cdbb8a157bcef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1e188af41ad458dbd316c405d8c0ecc in term 1.
I20260619 20:36:40.250990 30764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d1d1414813842b987bfa5b863c1542e" candidate_uuid: "e1e188af41ad458dbd316c405d8c0ecc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d010684da6f4ab09b7eb9f473926a12"
I20260619 20:36:40.251078 30764 raft_consensus.cc:3037] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:40.251104 30569 leader_election.cc:302] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 508d22e5059c4c5f9f5cdbb8a157bcef, e1e188af41ad458dbd316c405d8c0ecc; no voters:
I20260619 20:36:40.251176 30962 raft_consensus.cc:2781] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:40.251303 30962 raft_consensus.cc:686] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc [term 1 LEADER]: Becoming Leader. State: Replica: e1e188af41ad458dbd316c405d8c0ecc, State: Running, Role: LEADER
I20260619 20:36:40.251413 30962 consensus_queue.cc:237] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d010684da6f4ab09b7eb9f473926a12" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 40865 } } peers { permanent_uuid: "e1e188af41ad458dbd316c405d8c0ecc" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40131 } } peers { permanent_uuid: "508d22e5059c4c5f9f5cdbb8a157bcef" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 33079 } }
I20260619 20:36:40.251971 30764 raft_consensus.cc:2454] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1e188af41ad458dbd316c405d8c0ecc in term 1.
I20260619 20:36:40.252234 30491 catalog_manager.cc:5495] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc reported cstate change: term changed from 0 to 1, leader changed from <none> to e1e188af41ad458dbd316c405d8c0ecc (127.25.62.129). New cstate: current_term: 1 leader_uuid: "e1e188af41ad458dbd316c405d8c0ecc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d010684da6f4ab09b7eb9f473926a12" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 40865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1e188af41ad458dbd316c405d8c0ecc" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40131 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "508d22e5059c4c5f9f5cdbb8a157bcef" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 33079 } health_report { overall_health: UNKNOWN } } }
I20260619 20:36:40.277987 30952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.277097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50104 (local address 127.25.62.129:40131)
0619 20:36:40.277157 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:40.277159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:40.277168 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:36:40.277230 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:40.277230 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:40.277241 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:40.277295 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:40.277829 (+ 534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:40.277830 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:40.277830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:40.277832 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:40.277842 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:40.277845 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:40.277886 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:40.277916 (+ 30us) server_negotiation.cc:299] Negotiation successful
0619 20:36:40.277926 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 20:36:40.279631 30895 raft_consensus.cc:1261] T 1d1d1414813842b987bfa5b863c1542e P 508d22e5059c4c5f9f5cdbb8a157bcef [term 1 FOLLOWER]: Refusing update from remote peer e1e188af41ad458dbd316c405d8c0ecc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:40.279659 30764 raft_consensus.cc:1261] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12 [term 1 FOLLOWER]: Refusing update from remote peer e1e188af41ad458dbd316c405d8c0ecc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:40.280049 30965 consensus_queue.cc:1035] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc [LEADER]: Connected to new peer: Peer: permanent_uuid: "508d22e5059c4c5f9f5cdbb8a157bcef" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 33079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:40.280264 30962 consensus_queue.cc:1035] T 1d1d1414813842b987bfa5b863c1542e P e1e188af41ad458dbd316c405d8c0ecc [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d010684da6f4ab09b7eb9f473926a12" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 40865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:40.282711 30970 mvcc.cc:205] Tried to move back new op lower bound from 7298668135542865920 to 7298668135430438912. Current Snapshot: MvccSnapshot[applied={T|T < 7298668135542865920}]
I20260619 20:36:40.282829 30969 mvcc.cc:205] Tried to move back new op lower bound from 7298668135542865920 to 7298668135430438912. Current Snapshot: MvccSnapshot[applied={T|T < 7298668135542865920}]
I20260619 20:36:40.285363 30490 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:40.285468 30490 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:40.286285 30490 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 77f74961d1214000a28852eee810e8d4: Sending DeleteTablet for 3 replicas of tablet 1d1d1414813842b987bfa5b863c1542e
I20260619 20:36:40.286608 30613 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d1d1414813842b987bfa5b863c1542e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:40 UTC) from {username='slave'} at 127.0.0.1:50094
I20260619 20:36:40.286685 30743 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d1d1414813842b987bfa5b863c1542e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:40 UTC) from {username='slave'} at 127.0.0.1:51516
I20260619 20:36:40.286881 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 30551
I20260619 20:36:40.286913 30875 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d1d1414813842b987bfa5b863c1542e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:40 UTC) from {username='slave'} at 127.0.0.1:37990
I20260619 20:36:40.287159 30979 tablet_replica.cc:330] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12: stopping tablet replica
I20260619 20:36:40.287245 30979 raft_consensus.cc:2229] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:40.287403 30979 raft_consensus.cc:2258] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:40.287842 30979 ts_tablet_manager.cc:1905] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:40.290078 30979 ts_tablet_manager.cc:1918] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:36:40.290162 30979 log.cc:1196] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/1d1d1414813842b987bfa5b863c1542e
I20260619 20:36:40.290408 30979 ts_tablet_manager.cc:1939] T 1d1d1414813842b987bfa5b863c1542e P 5d010684da6f4ab09b7eb9f473926a12: Deleting consensus metadata
I20260619 20:36:40.290843 30476 catalog_manager.cc:4841] TS 5d010684da6f4ab09b7eb9f473926a12 (127.25.62.130:40865): tablet 1d1d1414813842b987bfa5b863c1542e (table test-table [id=f7a39ea1fc9c465a8cd4f70e3630965d]) successfully deleted
W20260619 20:36:40.292702 30478 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.62.129:40131 (error 108)
I20260619 20:36:40.292936 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 30682
I20260619 20:36:40.292968 30980 tablet_replica.cc:330] T 1d1d1414813842b987bfa5b863c1542e P 508d22e5059c4c5f9f5cdbb8a157bcef: stopping tablet replica
I20260619 20:36:40.293485 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.293071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:40131 (local address 127.0.0.1:50106)
0619 20:36:40.293403 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:40.293444 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.129:40131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313}
W20260619 20:36:40.294132 30478 catalog_manager.cc:4584] TS e1e188af41ad458dbd316c405d8c0ecc (127.25.62.129:40131): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1d1d1414813842b987bfa5b863c1542e: Network error: Client connection negotiation failed: client connection to 127.25.62.129:40131: connect: Connection refused (error 111)
I20260619 20:36:40.297204 30980 raft_consensus.cc:2229] T 1d1d1414813842b987bfa5b863c1542e P 508d22e5059c4c5f9f5cdbb8a157bcef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:40.297356 30980 raft_consensus.cc:2258] T 1d1d1414813842b987bfa5b863c1542e P 508d22e5059c4c5f9f5cdbb8a157bcef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:40.297520 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 30813
I20260619 20:36:40.297802 30980 ts_tablet_manager.cc:1905] T 1d1d1414813842b987bfa5b863c1542e P 508d22e5059c4c5f9f5cdbb8a157bcef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:40.301998 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.301885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:33079 (local address 127.0.0.1:38004)
0619 20:36:40.301934 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:40.301958 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.131:33079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 20:36:40.302418 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 30460
2026-06-19T20:36:40Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (691 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 20:36:40.322335 25850 mini_oidc.cc:152] Starting JWKS server
I20260619 20:36:40.323426 25850 webserver.cc:460] Webserver started at https://127.0.0.1:43739/ using document root <none> and password file <none>
I20260619 20:36:40.323556 25850 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 20:36:40.323751 25850 webserver.cc:460] Webserver started at http://127.0.0.1:35181/ using document root <none> and password file <none>
2026-06-19T20:36:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:40Z Disabled control of system clock
I20260619 20:36:40.331501 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:38075
--webserver_interface=127.25.62.190
--webserver_port=0
--builtin_ntp_servers=127.25.62.148:42169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:38075
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43739/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0/testchainca.cert with env {}
W20260619 20:36:40.412894 30987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:40.413125 30987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:40.413151 30987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:40.414364 30987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:40.414413 30987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:40.414428 30987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:40.414443 30987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:40.414456 30987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 20:36:40.414516 30987 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 20:36:40.414535 30987 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43739/jwks.json
I20260619 20:36:40.416144 30987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:42169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:38075
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:38075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43739/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:40.416422 30987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:40.416658 30987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:40.421869 30995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:40.421888 30993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:40.421888 30992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:40.422210 30987 server_base.cc:1034] running on GCE node
I20260619 20:36:40.422516 30987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:40.422811 30987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:40.423985 30987 hybrid_clock.cc:648] HybridClock initialized: now 1781901400423918 us; error 79 us; skew 500 ppm
I20260619 20:36:40.425630 30987 webserver.cc:460] Webserver started at http://127.25.62.190:40245/ using document root <none> and password file <none>
I20260619 20:36:40.425850 30987 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:40.425901 30987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:40.426000 30987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:40.427107 30987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "5ede041a4d994d16ab82342ee2f24429"
format_stamp: "Formatted at 2026-06-19 20:36:40 on dist-test-slave-0t40"
server_key: "88f0cab32d3344fe128f3dc59bbce27d"
server_key_iv: "4fd0665363b6af9229b9f9cecb8f1125"
server_key_version: "encryptionkey@0"
I20260619 20:36:40.427450 30987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "5ede041a4d994d16ab82342ee2f24429"
format_stamp: "Formatted at 2026-06-19 20:36:40 on dist-test-slave-0t40"
server_key: "88f0cab32d3344fe128f3dc59bbce27d"
server_key_iv: "4fd0665363b6af9229b9f9cecb8f1125"
server_key_version: "encryptionkey@0"
I20260619 20:36:40.428949 30987 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:36:40.429881 31001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:40.430119 30987 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:40.430210 30987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "5ede041a4d994d16ab82342ee2f24429"
format_stamp: "Formatted at 2026-06-19 20:36:40 on dist-test-slave-0t40"
server_key: "88f0cab32d3344fe128f3dc59bbce27d"
server_key_iv: "4fd0665363b6af9229b9f9cecb8f1125"
server_key_version: "encryptionkey@0"
I20260619 20:36:40.430274 30987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:40.448643 30987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:40.448906 30987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:40.449018 30987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:40.452785 30987 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:38075
I20260619 20:36:40.452858 31053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:38075 every 8 connection(s)
I20260619 20:36:40.453226 30987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:40.454208 31054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:40.455912 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 30987
I20260619 20:36:40.456020 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:36:40.456193 25850 external_mini_cluster.cc:1428] Setting key a2dae09907196ed438a517efb196c857
I20260619 20:36:40.456894 31054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429: Bootstrap starting.
I20260619 20:36:40.457533 31054 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:40.457841 31054 log.cc:824] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:40.458568 31054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429: No bootstrap required, opened a new log
I20260619 20:36:40.460713 31054 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ede041a4d994d16ab82342ee2f24429" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 38075 } }
I20260619 20:36:40.460899 31054 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:40.460939 31054 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ede041a4d994d16ab82342ee2f24429, State: Initialized, Role: FOLLOWER
I20260619 20:36:40.461110 31054 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ede041a4d994d16ab82342ee2f24429" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 38075 } }
I20260619 20:36:40.461179 31054 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:40.461201 31054 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:40.461253 31054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:40.461854 31054 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ede041a4d994d16ab82342ee2f24429" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 38075 } }
I20260619 20:36:40.461977 31054 leader_election.cc:302] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ede041a4d994d16ab82342ee2f24429; no voters:
I20260619 20:36:40.462136 31054 leader_election.cc:288] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:40.462318 31060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:40.462437 31054 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:40.462522 31060 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429 [term 1 LEADER]: Becoming Leader. State: Replica: 5ede041a4d994d16ab82342ee2f24429, State: Running, Role: LEADER
I20260619 20:36:40.462646 31060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ede041a4d994d16ab82342ee2f24429" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 38075 } }
I20260619 20:36:40.462829 31057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.456691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33196 (local address 127.25.62.190:38075)
0619 20:36:40.456891 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:40.456899 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:40.456999 (+ 100us) server_negotiation.cc:407] Connection header received
0619 20:36:40.457464 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:40.457470 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:40.457531 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:40.457687 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:40.457998 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:40.459305 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:40.460120 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:40.460802 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:40.460940 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:40.461749 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:40.461754 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:40.461754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:40.461966 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:40.462010 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:40.462026 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:40.462132 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:40.462416 (+ 284us) server_negotiation.cc:299] Negotiation successful
0619 20:36:40.462506 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260619 20:36:40.462971 31059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ede041a4d994d16ab82342ee2f24429. Latest consensus state: current_term: 1 leader_uuid: "5ede041a4d994d16ab82342ee2f24429" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ede041a4d994d16ab82342ee2f24429" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 38075 } } }
I20260619 20:36:40.463064 31059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:40.463227 31061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ede041a4d994d16ab82342ee2f24429" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ede041a4d994d16ab82342ee2f24429" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 38075 } } }
I20260619 20:36:40.463321 31061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:40.463850 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:36:40.464042 31063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:40.464524 31063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:40.466094 31063 catalog_manager.cc:1347] Generated new cluster ID: 3244b18b717448818455bd0ec5d0be42
I20260619 20:36:40.466136 31063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:40.488015 31063 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:40.488571 31063 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:40.495832 31063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ede041a4d994d16ab82342ee2f24429: Generated new TSK 0
I20260619 20:36:40.496022 31063 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 20:36:40.522101 31084 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 20:36:40.522199 31084 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 20:36:40.522485 31057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.506942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33202 (local address 127.25.62.190:38075)
0619 20:36:40.507003 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:40.507005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:40.507139 (+ 134us) server_negotiation.cc:407] Connection header received
0619 20:36:40.507263 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:40.507264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:40.507277 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:40.507335 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0619 20:36:40.507530 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:40.508007 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:40.509147 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:40.509331 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:40.509388 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:40.509620 (+ 232us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 20:36:40.522240 (+ 12620us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43739/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 20:36:40.522410 (+ 170us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33202: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43739/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":36}
W20260619 20:36:40.522552 31057 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33202: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43739/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260619 20:36:40.522807 31083 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:40.506870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:38075 (local address 127.0.0.1:33202)
0619 20:36:40.507074 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:40.507086 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:40.507164 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:40.507377 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:40.507378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:40.507380 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 20:36:40.507458 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:40.507461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:40.508103 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:40.508105 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:40.509033 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:40.509063 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:40.509416 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:40.509418 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:40.509492 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:40.509516 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 20:36:40.522391 (+ 12875us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43739/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 20:36:40.522483 (+ 92us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.62.190:38075: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43739/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.25.62.190:38075: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43739/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 20:36:40.523279 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 30987
2026-06-19T20:36:40Z chronyd exiting
I20260619 20:36:40.559219 25850 test_util.cc:175] -----------------------------------------------
I20260619 20:36:40.559296 25850 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781901378226483-25850-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (237 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 20:36:40.560045 25850 mini_oidc.cc:152] Starting JWKS server
I20260619 20:36:40.560741 25850 webserver.cc:460] Webserver started at https://127.0.0.1:42337/ using document root <none> and password file <none>
I20260619 20:36:40.560781 25850 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 20:36:40.560961 25850 webserver.cc:460] Webserver started at http://127.0.0.1:45343/ using document root <none> and password file <none>
2026-06-19T20:36:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:40Z Disabled control of system clock
I20260619 20:36:40.568320 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:37981
--webserver_interface=127.25.62.190
--webserver_port=0
--builtin_ntp_servers=127.25.62.148:45815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:37981
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42337/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901378226483-25850-0/testchainca.cert with env {}
W20260619 20:36:40.647886 31090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:40.648085 31090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:40.648108 31090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:40.649211 31090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:40.649246 31090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:40.649259 31090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:40.649271 31090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:40.649283 31090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 20:36:40.649295 31090 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 20:36:40.649312 31090 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42337/jwks.json
I20260619 20:36:40.650751 31090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:45815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:37981
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:37981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42337/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901378226483-25850-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:40.650939 31090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:40.651095 31090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:40.655925 31098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:40.656020 31090 server_base.cc:1034] running on GCE node
W20260619 20:36:40.655977 31095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:40.655977 31096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:40.656386 31090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:40.656682 31090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:40.657838 31090 hybrid_clock.cc:648] HybridClock initialized: now 1781901400657823 us; error 40 us; skew 500 ppm
I20260619 20:36:40.659053 31090 webserver.cc:460] Webserver started at http://127.25.62.190:36243/ using document root <none> and password file <none>
I20260619 20:36:40.659293 31090 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:40.659364 31090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:40.659516 31090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:40.660998 31090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "1b78e25ab337457daa2abe4aa273e742"
format_stamp: "Formatted at 2026-06-19 20:36:40 on dist-test-slave-0t40"
server_key: "9af82c3ab0a1100c7f356588b25c8535"
server_key_iv: "1c801abfc6469b69c5a97216890affb5"
server_key_version: "encryptionkey@0"
I20260619 20:36:40.661480 31090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "1b78e25ab337457daa2abe4aa273e742"
format_stamp: "Formatted at 2026-06-19 20:36:40 on dist-test-slave-0t40"
server_key: "9af82c3ab0a1100c7f356588b25c8535"
server_key_iv: "1c801abfc6469b69c5a97216890affb5"
server_key_version: "encryptionkey@0"
I20260619 20:36:40.663352 31090 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 20:36:40.665107 31104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:40.665352 31090 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:40.665483 31090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "1b78e25ab337457daa2abe4aa273e742"
format_stamp: "Formatted at 2026-06-19 20:36:40 on dist-test-slave-0t40"
server_key: "9af82c3ab0a1100c7f356588b25c8535"
server_key_iv: "1c801abfc6469b69c5a97216890affb5"
server_key_version: "encryptionkey@0"
I20260619 20:36:40.665623 31090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:40.706142 31090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:40.706496 31090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:40.706653 31090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:40.710467 31090 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:37981
I20260619 20:36:40.710526 31156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:37981 every 8 connection(s)
I20260619 20:36:40.710858 31090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:40.711534 31157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:40.712553 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 31090
I20260619 20:36:40.712669 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:36:40.712868 25850 external_mini_cluster.cc:1428] Setting key b0d206109a8b3a26551f4fa29876af1f
I20260619 20:36:40.715548 31157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742: Bootstrap starting.
I20260619 20:36:40.716234 31157 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:40.716548 31157 log.cc:824] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:40.717325 31157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742: No bootstrap required, opened a new log
I20260619 20:36:40.718474 31160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.713419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57304 (local address 127.25.62.190:37981)
0619 20:36:40.713597 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:40.713605 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:40.713635 (+ 30us) server_negotiation.cc:407] Connection header received
0619 20:36:40.714055 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:40.714062 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:40.714122 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:40.714235 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:40.714461 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:40.715343 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:40.716433 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:40.716866 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:40.717013 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:40.717822 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:40.717827 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:40.717827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:40.718032 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:40.718064 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:40.718075 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:40.718147 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:40.718233 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 20:36:40.718315 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260619 20:36:40.719435 31157 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b78e25ab337457daa2abe4aa273e742" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37981 } }
I20260619 20:36:40.719609 31157 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:40.719622 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:36:40.719645 31157 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b78e25ab337457daa2abe4aa273e742, State: Initialized, Role: FOLLOWER
I20260619 20:36:40.719758 31157 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b78e25ab337457daa2abe4aa273e742" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37981 } }
I20260619 20:36:40.719825 31157 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:40.719866 31157 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:40.719914 31157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:40.720602 31157 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b78e25ab337457daa2abe4aa273e742" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37981 } }
I20260619 20:36:40.720713 31157 leader_election.cc:302] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b78e25ab337457daa2abe4aa273e742; no voters:
I20260619 20:36:40.720863 31157 leader_election.cc:288] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:40.720927 31162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:40.721115 31162 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742 [term 1 LEADER]: Becoming Leader. State: Replica: 1b78e25ab337457daa2abe4aa273e742, State: Running, Role: LEADER
I20260619 20:36:40.721127 31157 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:40.721220 31162 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b78e25ab337457daa2abe4aa273e742" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37981 } }
I20260619 20:36:40.721724 31163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b78e25ab337457daa2abe4aa273e742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b78e25ab337457daa2abe4aa273e742" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37981 } } }
I20260619 20:36:40.721724 31164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b78e25ab337457daa2abe4aa273e742. Latest consensus state: current_term: 1 leader_uuid: "1b78e25ab337457daa2abe4aa273e742" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b78e25ab337457daa2abe4aa273e742" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37981 } } }
I20260619 20:36:40.721835 31163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:40.721843 31164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:40.722034 31170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:40.722491 31170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:40.723949 31170 catalog_manager.cc:1347] Generated new cluster ID: cbe6520a2a6b4ceabe7aa4bc0c3163fb
I20260619 20:36:40.724000 31170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:40.738255 31170 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:40.738778 31170 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:40.741454 31170 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b78e25ab337457daa2abe4aa273e742: Generated new TSK 0
I20260619 20:36:40.741626 31170 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 20:36:40.762217 31187 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 20:36:40.762313 31187 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 20:36:40.762660 31160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.747951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57310 (local address 127.25.62.190:37981)
0619 20:36:40.748014 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:40.748016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:40.748067 (+ 51us) server_negotiation.cc:407] Connection header received
0619 20:36:40.748145 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:40.748146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:40.748156 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:40.748212 (+ 56us) server_negotiation.cc:226] Negotiated authn=JWT
0619 20:36:40.748354 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:40.748832 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:40.749992 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:40.750176 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:40.750229 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:40.750458 (+ 229us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 20:36:40.762469 (+ 12011us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42337/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 20:36:40.762601 (+ 132us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57310: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42337/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":37}
W20260619 20:36:40.762733 31160 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57310: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42337/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260619 20:36:40.762887 31186 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:40.747869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37981 (local address 127.0.0.1:57310)
0619 20:36:40.748016 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:40.748026 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:40.748082 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:40.748224 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:40.748225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:40.748227 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 20:36:40.748294 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:40.748296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:40.748911 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:40.748912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:40.749906 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:40.749910 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:40.750253 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:40.750254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:40.750315 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:40.750321 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 20:36:40.762590 (+ 12269us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42337/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 20:36:40.762775 (+ 185us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.62.190:37981: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42337/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260619 20:36:40.763185 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 31090
2026-06-19T20:36:40Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (240 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 20:36:40.800575 25850 mini_oidc.cc:152] Starting JWKS server
I20260619 20:36:40.801632 25850 webserver.cc:460] Webserver started at https://127.0.0.1:43745/ using document root <none> and password file <none>
I20260619 20:36:40.801715 25850 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 20:36:40.801899 25850 webserver.cc:460] Webserver started at http://127.0.0.1:46423/ using document root <none> and password file <none>
2026-06-19T20:36:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:40Z Disabled control of system clock
I20260619 20:36:40.809823 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:33125
--webserver_interface=127.25.62.190
--webserver_port=0
--builtin_ntp_servers=127.25.62.148:34609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:33125
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43745/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 20:36:40.900538 31193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:40.900748 31193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:40.900772 31193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:40.901978 31193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:40.902024 31193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:40.902040 31193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:40.902053 31193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:40.902067 31193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 20:36:40.902082 31193 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 20:36:40.902096 31193 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43745/jwks.json
I20260619 20:36:40.903386 31193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:34609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:33125
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:33125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43745/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:40.903579 31193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:40.903810 31193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:40.908599 31198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:40.908634 31199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:40.908699 31193 server_base.cc:1034] running on GCE node
W20260619 20:36:40.908649 31201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:40.909130 31193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:40.909431 31193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:40.910593 31193 hybrid_clock.cc:648] HybridClock initialized: now 1781901400910588 us; error 39 us; skew 500 ppm
I20260619 20:36:40.911793 31193 webserver.cc:460] Webserver started at http://127.25.62.190:42131/ using document root <none> and password file <none>
I20260619 20:36:40.912043 31193 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:40.912110 31193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:40.912256 31193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:40.913362 31193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "6e5bceb5d87f44ee9b81f27f0b8ff8a0"
format_stamp: "Formatted at 2026-06-19 20:36:40 on dist-test-slave-0t40"
server_key: "0ec5cbee352371257bfdc761b685efa7"
server_key_iv: "b8c5ac9886281d44d04c3bed69c11219"
server_key_version: "encryptionkey@0"
I20260619 20:36:40.913710 31193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "6e5bceb5d87f44ee9b81f27f0b8ff8a0"
format_stamp: "Formatted at 2026-06-19 20:36:40 on dist-test-slave-0t40"
server_key: "0ec5cbee352371257bfdc761b685efa7"
server_key_iv: "b8c5ac9886281d44d04c3bed69c11219"
server_key_version: "encryptionkey@0"
I20260619 20:36:40.915014 31193 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:36:40.915799 31207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:40.916028 31193 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:40.916133 31193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "6e5bceb5d87f44ee9b81f27f0b8ff8a0"
format_stamp: "Formatted at 2026-06-19 20:36:40 on dist-test-slave-0t40"
server_key: "0ec5cbee352371257bfdc761b685efa7"
server_key_iv: "b8c5ac9886281d44d04c3bed69c11219"
server_key_version: "encryptionkey@0"
I20260619 20:36:40.916208 31193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:40.931350 31193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:40.931651 31193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:40.931799 31193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:40.935556 31193 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:33125
I20260619 20:36:40.935598 31259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:33125 every 8 connection(s)
I20260619 20:36:40.935951 31193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:40.936419 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 31193
I20260619 20:36:40.936522 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:36:40.936700 25850 external_mini_cluster.cc:1428] Setting key 24efe1c41f095b0f51d7ed4b9cafc58d
I20260619 20:36:40.936702 31260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:40.940208 31260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0: Bootstrap starting.
I20260619 20:36:40.940860 31260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:40.941241 31260 log.cc:824] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:40.942008 31260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0: No bootstrap required, opened a new log
I20260619 20:36:40.942544 31263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.937268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40672 (local address 127.25.62.190:33125)
0619 20:36:40.937479 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:40.937486 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:40.937509 (+ 23us) server_negotiation.cc:407] Connection header received
0619 20:36:40.938250 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:40.938256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:40.938316 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:40.938437 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:40.938699 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:40.939524 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:40.940606 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:40.941046 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:40.941170 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:40.941891 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:40.941895 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:40.941896 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:40.942079 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:40.942121 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:40.942139 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:40.942188 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:40.942269 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 20:36:40.942346 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
I20260619 20:36:40.943467 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:36:40.944093 31260 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e5bceb5d87f44ee9b81f27f0b8ff8a0" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 33125 } }
I20260619 20:36:40.944259 31260 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:40.944286 31260 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e5bceb5d87f44ee9b81f27f0b8ff8a0, State: Initialized, Role: FOLLOWER
I20260619 20:36:40.944437 31260 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e5bceb5d87f44ee9b81f27f0b8ff8a0" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 33125 } }
I20260619 20:36:40.944519 31260 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:40.944540 31260 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:40.944602 31260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:40.945166 31260 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e5bceb5d87f44ee9b81f27f0b8ff8a0" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 33125 } }
I20260619 20:36:40.945247 31260 leader_election.cc:302] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e5bceb5d87f44ee9b81f27f0b8ff8a0; no voters:
I20260619 20:36:40.945403 31260 leader_election.cc:288] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:40.945492 31265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:40.945638 31260 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:40.945690 31265 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0 [term 1 LEADER]: Becoming Leader. State: Replica: 6e5bceb5d87f44ee9b81f27f0b8ff8a0, State: Running, Role: LEADER
I20260619 20:36:40.945786 31265 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e5bceb5d87f44ee9b81f27f0b8ff8a0" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 33125 } }
I20260619 20:36:40.946221 31266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e5bceb5d87f44ee9b81f27f0b8ff8a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e5bceb5d87f44ee9b81f27f0b8ff8a0" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 33125 } } }
I20260619 20:36:40.946257 31267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e5bceb5d87f44ee9b81f27f0b8ff8a0. Latest consensus state: current_term: 1 leader_uuid: "6e5bceb5d87f44ee9b81f27f0b8ff8a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e5bceb5d87f44ee9b81f27f0b8ff8a0" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 33125 } } }
I20260619 20:36:40.946336 31266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:40.946341 31267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0 [sys.catalog]: This master's current role is: LEADER
W20260619 20:36:40.947431 31281 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:36:40.947499 31281 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:36:40.947580 31274 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:40.947764 31274 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:40.949193 31274 catalog_manager.cc:1347] Generated new cluster ID: 949f61a744434b13bd6251d3b784ba83
I20260619 20:36:40.949245 31274 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:40.964757 31274 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:40.965533 31274 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:40.969249 31274 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e5bceb5d87f44ee9b81f27f0b8ff8a0: Generated new TSK 0
I20260619 20:36:40.969434 31274 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:36:40.981145 31263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:40.977372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40678 (local address 127.25.62.190:33125)
0619 20:36:40.977417 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:40.977420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:40.977531 (+ 111us) server_negotiation.cc:407] Connection header received
0619 20:36:40.977594 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:40.977594 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:40.977607 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:40.977684 (+ 77us) server_negotiation.cc:226] Negotiated authn=JWT
0619 20:36:40.977803 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:40.978602 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:40.979404 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:40.979729 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:40.979799 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:40.979969 (+ 170us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 20:36:40.980907 (+ 938us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43745/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 20:36:40.981064 (+ 157us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40678: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43745/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":24}
W20260619 20:36:40.981205 31263 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40678: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43745/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 20:36:40.983042 31289 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:40.977282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:33125 (local address 127.0.0.1:40678)
0619 20:36:40.977447 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:40.977458 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:40.977524 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:40.977683 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:40.977684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:40.977685 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0619 20:36:40.977734 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:40.977736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:40.978696 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:40.978697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:40.979314 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:40.979316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:40.979811 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:40.979812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:40.979856 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:40.979861 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 20:36:40.981010 (+ 1149us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43745/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 20:36:40.982935 (+ 1925us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.62.190:33125: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43745/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":1728,"thread_start_us":116,"threads_started":1}
I20260619 20:36:40.983278 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 31193
E20260619 20:36:40.992112 31290 webserver.cc:533] Webserver: sslize failed:
E20260619 20:36:40.992187 31290 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-19T20:36:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (220 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T20:36:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:41Z Disabled control of system clock
I20260619 20:36:41.028092 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:33311
--webserver_interface=127.25.62.190
--webserver_port=0
--builtin_ntp_servers=127.25.62.148:35357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:33311
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901378226483-25850-0/testchainca.cert with env {}
W20260619 20:36:41.107326 31294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:41.107525 31294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:41.107547 31294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:41.108704 31294 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:41.108742 31294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:41.108755 31294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:41.108768 31294 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:41.108780 31294 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 20:36:41.108830 31294 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 20:36:41.108847 31294 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 20:36:41.110301 31294 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:35357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:33311
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:33311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901378226483-25850-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:41.110556 31294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:41.110769 31294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:41.115468 31300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:41.115476 31299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:41.115641 31294 server_base.cc:1034] running on GCE node
W20260619 20:36:41.115485 31302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:41.115950 31294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:41.116267 31294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:41.117419 31294 hybrid_clock.cc:648] HybridClock initialized: now 1781901401117409 us; error 42 us; skew 500 ppm
I20260619 20:36:41.118729 31294 webserver.cc:460] Webserver started at http://127.25.62.190:39645/ using document root <none> and password file <none>
I20260619 20:36:41.118966 31294 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:41.119035 31294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:41.119167 31294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:41.120177 31294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "0ff0dbee9efb4325ac1d73b35ec15baf"
format_stamp: "Formatted at 2026-06-19 20:36:41 on dist-test-slave-0t40"
server_key: "0eea249a8588ddb4cfd6922ca4b91707"
server_key_iv: "575a95ee49eb323c470e8fccce2e13ce"
server_key_version: "encryptionkey@0"
I20260619 20:36:41.120527 31294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "0ff0dbee9efb4325ac1d73b35ec15baf"
format_stamp: "Formatted at 2026-06-19 20:36:41 on dist-test-slave-0t40"
server_key: "0eea249a8588ddb4cfd6922ca4b91707"
server_key_iv: "575a95ee49eb323c470e8fccce2e13ce"
server_key_version: "encryptionkey@0"
I20260619 20:36:41.121886 31294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:36:41.122632 31308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:41.122818 31294 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:41.122915 31294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "0ff0dbee9efb4325ac1d73b35ec15baf"
format_stamp: "Formatted at 2026-06-19 20:36:41 on dist-test-slave-0t40"
server_key: "0eea249a8588ddb4cfd6922ca4b91707"
server_key_iv: "575a95ee49eb323c470e8fccce2e13ce"
server_key_version: "encryptionkey@0"
I20260619 20:36:41.122982 31294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:41.137604 31294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:41.137897 31294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:41.138044 31294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:41.141957 31294 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:33311
I20260619 20:36:41.141991 31360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:33311 every 8 connection(s)
I20260619 20:36:41.142320 31294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:41.143221 31361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:41.145375 31361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf: Bootstrap starting.
I20260619 20:36:41.145926 31361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:41.146211 31361 log.cc:824] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:41.146854 31361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf: No bootstrap required, opened a new log
I20260619 20:36:41.148396 31361 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff0dbee9efb4325ac1d73b35ec15baf" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 33311 } }
I20260619 20:36:41.148536 31361 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:41.148558 31361 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ff0dbee9efb4325ac1d73b35ec15baf, State: Initialized, Role: FOLLOWER
I20260619 20:36:41.148706 31361 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff0dbee9efb4325ac1d73b35ec15baf" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 33311 } }
I20260619 20:36:41.148766 31361 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:41.148787 31361 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:41.148854 31361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:41.149425 31361 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff0dbee9efb4325ac1d73b35ec15baf" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 33311 } }
I20260619 20:36:41.149513 31361 leader_election.cc:302] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ff0dbee9efb4325ac1d73b35ec15baf; no voters:
I20260619 20:36:41.149696 31361 leader_election.cc:288] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:41.149737 31364 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:41.149909 31364 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf [term 1 LEADER]: Becoming Leader. State: Replica: 0ff0dbee9efb4325ac1d73b35ec15baf, State: Running, Role: LEADER
I20260619 20:36:41.149938 31361 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:41.150004 31364 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff0dbee9efb4325ac1d73b35ec15baf" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 33311 } }
I20260619 20:36:41.150514 31365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ff0dbee9efb4325ac1d73b35ec15baf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff0dbee9efb4325ac1d73b35ec15baf" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 33311 } } }
I20260619 20:36:41.150599 31366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ff0dbee9efb4325ac1d73b35ec15baf. Latest consensus state: current_term: 1 leader_uuid: "0ff0dbee9efb4325ac1d73b35ec15baf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff0dbee9efb4325ac1d73b35ec15baf" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 33311 } } }
I20260619 20:36:41.150691 31366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:41.150611 31365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:41.151093 31373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:41.151582 31373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:41.152089 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 31294
I20260619 20:36:41.152195 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:36:41.152366 25850 external_mini_cluster.cc:1428] Setting key 24c00eb0afa2f79ee5fcb8068e933d2d
I20260619 20:36:41.153017 31373 catalog_manager.cc:1347] Generated new cluster ID: d7b3ecc261ba4c438bdd5d7583229705
I20260619 20:36:41.153106 31373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:41.161365 31384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:41.152739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57082 (local address 127.25.62.190:33311)
0619 20:36:41.157225 (+ 4486us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:41.157234 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:41.157271 (+ 37us) server_negotiation.cc:407] Connection header received
0619 20:36:41.157357 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:41.157362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:41.157412 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:41.157560 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:41.157768 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:41.158672 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:41.159455 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:41.159869 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:41.159992 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:41.160692 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:41.160696 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:41.160696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:36:41.160877 (+ 181us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:41.160915 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:36:41.160926 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:41.161004 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:41.161093 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 20:36:41.161173 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4408,"thread_start_us":67,"threads_started":1}
I20260619 20:36:41.162163 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:36:41.170980 31373 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:41.171636 31373 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:41.193603 31373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ff0dbee9efb4325ac1d73b35ec15baf: Generated new TSK 0
I20260619 20:36:41.193797 31373 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:36:41.285382 31384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:41.204565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57092 (local address 127.25.62.190:33311)
0619 20:36:41.204630 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:41.204632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:41.204769 (+ 137us) server_negotiation.cc:407] Connection header received
0619 20:36:41.204841 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:41.204842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:41.204853 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:41.204917 (+ 64us) server_negotiation.cc:226] Negotiated authn=JWT
0619 20:36:41.205101 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:41.205599 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:41.206742 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:41.206939 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:41.206989 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:41.207207 (+ 218us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 20:36:41.285170 (+ 77963us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 20:36:41.285325 (+ 155us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57092: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":38}
W20260619 20:36:41.285475 31384 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57092: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 20:36:41.285722 31390 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:41.204511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:33311 (local address 127.0.0.1:57092)
0619 20:36:41.204674 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:41.204686 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:41.204763 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:41.204927 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:41.204928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:41.204930 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 20:36:41.204995 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:41.204998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:41.205700 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:41.205701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:41.206639 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:41.206642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:41.207019 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:41.207020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:41.207083 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:41.207089 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 20:36:41.285327 (+ 78238us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 20:36:41.285505 (+ 178us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.25.62.190:33311: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":106,"threads_started":1}
I20260619 20:36:41.286069 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 31294
2026-06-19T20:36:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (282 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (87 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (86 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:41 dist-test-slave-0t40 krb5kdc[31402](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:41 dist-test-slave-0t40 krb5kdc[31402](info): set up 2 sockets
Jun 19 20:36:41 dist-test-slave-0t40 krb5kdc[31402](info): commencing operation
krb5kdc: starting...
W20260619 20:36:42.501446 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:36:42 dist-test-slave-0t40 krb5kdc[31402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:36:42.618088 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:35175
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:35175
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:42.702343 31415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:42.702533 31415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:42.702555 31415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:42.703708 31415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:42.703750 31415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:42.703764 31415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:42.703778 31415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:42.703789 31415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:42.705252 31415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:35175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:35175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:42.705489 31415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:42.705710 31415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:42.710431 31423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:42.710480 31415 server_base.cc:1034] running on GCE node
W20260619 20:36:42.710444 31421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:42.710445 31420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:42.710824 31415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:42.711107 31415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:42.712265 31415 hybrid_clock.cc:648] HybridClock initialized: now 1781901402712254 us; error 42 us; skew 500 ppm
Jun 19 20:36:42 dist-test-slave-0t40 krb5kdc[31402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901402, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:42.714648 31415 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:36:42.715102 31415 webserver.cc:460] Webserver started at http://127.25.62.190:39673/ using document root <none> and password file <none>
I20260619 20:36:42.715335 31415 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:42.715399 31415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:42.715543 31415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:42.716554 31415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "9167c561ae3546a881f0e21b730f76d0"
format_stamp: "Formatted at 2026-06-19 20:36:42 on dist-test-slave-0t40"
server_key: "a395dfed74f6d1031336833a1f095003"
server_key_iv: "e8dc4eeab84cfd87f6919596ee666dd3"
server_key_version: "encryptionkey@0"
I20260619 20:36:42.716912 31415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "9167c561ae3546a881f0e21b730f76d0"
format_stamp: "Formatted at 2026-06-19 20:36:42 on dist-test-slave-0t40"
server_key: "a395dfed74f6d1031336833a1f095003"
server_key_iv: "e8dc4eeab84cfd87f6919596ee666dd3"
server_key_version: "encryptionkey@0"
I20260619 20:36:42.718437 31415 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:42.719157 31430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:42.719324 31415 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:36:42.719419 31415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "9167c561ae3546a881f0e21b730f76d0"
format_stamp: "Formatted at 2026-06-19 20:36:42 on dist-test-slave-0t40"
server_key: "a395dfed74f6d1031336833a1f095003"
server_key_iv: "e8dc4eeab84cfd87f6919596ee666dd3"
server_key_version: "encryptionkey@0"
I20260619 20:36:42.719501 31415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:42.731456 31415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:42.733510 31415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:42.733661 31415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:42.737601 31415 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:35175
I20260619 20:36:42.737628 31482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:35175 every 8 connection(s)
I20260619 20:36:42.737967 31415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:42.738785 31483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:42.740957 31483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0: Bootstrap starting.
I20260619 20:36:42.741554 31483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:42.741843 31483 log.cc:824] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:42.742504 31483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0: No bootstrap required, opened a new log
I20260619 20:36:42.742511 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 31415
I20260619 20:36:42.742607 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:36:42.742795 25850 external_mini_cluster.cc:1428] Setting key 89bff5c75edcfb29391ca91035237a29
I20260619 20:36:42.744385 31483 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9167c561ae3546a881f0e21b730f76d0" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35175 } }
I20260619 20:36:42.744554 31483 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:42.744592 31483 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9167c561ae3546a881f0e21b730f76d0, State: Initialized, Role: FOLLOWER
I20260619 20:36:42.744724 31483 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9167c561ae3546a881f0e21b730f76d0" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35175 } }
I20260619 20:36:42.744807 31483 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:42.744843 31483 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:42.744895 31483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:42.745908 31483 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9167c561ae3546a881f0e21b730f76d0" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35175 } }
I20260619 20:36:42.745993 31483 leader_election.cc:302] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9167c561ae3546a881f0e21b730f76d0; no voters:
I20260619 20:36:42.746184 31483 leader_election.cc:288] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:42.746232 31488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:42.746373 31488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0 [term 1 LEADER]: Becoming Leader. State: Replica: 9167c561ae3546a881f0e21b730f76d0, State: Running, Role: LEADER
I20260619 20:36:42.746464 31483 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:42.746451 31488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9167c561ae3546a881f0e21b730f76d0" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35175 } }
I20260619 20:36:42.746754 31488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9167c561ae3546a881f0e21b730f76d0. Latest consensus state: current_term: 1 leader_uuid: "9167c561ae3546a881f0e21b730f76d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9167c561ae3546a881f0e21b730f76d0" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35175 } } }
I20260619 20:36:42.746840 31488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:42.746762 31490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9167c561ae3546a881f0e21b730f76d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9167c561ae3546a881f0e21b730f76d0" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35175 } } }
I20260619 20:36:42.746996 31490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0 [sys.catalog]: This master's current role is: LEADER
W20260619 20:36:42.748133 31503 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:36:42.748188 31503 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:36:42.748353 31494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:42.748538 31494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 20:36:42 dist-test-slave-0t40 krb5kdc[31402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:42.754078 31487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:42.743304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58462 (local address 127.25.62.190:35175)
0619 20:36:42.743495 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:42.743502 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:42.743525 (+ 23us) server_negotiation.cc:407] Connection header received
0619 20:36:42.744367 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:42.744373 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:42.744419 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:42.744536 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:42.745392 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:42.746497 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:42.747545 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:42.748140 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:42.748252 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:42.750893 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:42.750897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:42.750898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:42.750916 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:42.752493 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:42.752868 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:42.752871 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:42.752871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:42.752981 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:42.753223 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:42.753224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:42.753224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:42.753461 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:42.753665 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:42.753738 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 20:36:42.753884 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":101,"threads_started":1}
I20260619 20:36:42.754218 31494 catalog_manager.cc:1347] Generated new cluster ID: 79f1e19f38464a5c8171618cbc8dc56b
I20260619 20:36:42.754253 31494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:42.764823 31494 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:42.765570 31494 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:42.772483 31494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9167c561ae3546a881f0e21b730f76d0: Generated new TSK 0
I20260619 20:36:42.772661 31494 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
I20260619 20:36:42.808414 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:35175
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:42.892495 31509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:42.892706 31509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:42.892733 31509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:42.893925 31509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:42.893971 31509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:42.893999 31509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:36:42.895428 31509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:35175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:42.895605 31509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:42.895773 31509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:42.900511 31514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:42.900503 31517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:42.900503 31515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:42.901234 31509 server_base.cc:1034] running on GCE node
I20260619 20:36:42.901482 31509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:42.901708 31509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:42.902881 31509 hybrid_clock.cc:648] HybridClock initialized: now 1781901402902859 us; error 30 us; skew 500 ppm
Jun 19 20:36:42 dist-test-slave-0t40 krb5kdc[31402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901402, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:42.905375 31509 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:36:42.905814 31509 webserver.cc:460] Webserver started at http://127.25.62.129:42397/ using document root <none> and password file <none>
I20260619 20:36:42.905992 31509 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:42.906029 31509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:42.906116 31509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:42.907100 31509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "a665f65a8d73405d8803aefbb2a7472c"
format_stamp: "Formatted at 2026-06-19 20:36:42 on dist-test-slave-0t40"
server_key: "c595b321322c6dc490870e1aecd6c263"
server_key_iv: "850618e7c488f523369a2f48ad975a83"
server_key_version: "encryptionkey@0"
I20260619 20:36:42.907397 31509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "a665f65a8d73405d8803aefbb2a7472c"
format_stamp: "Formatted at 2026-06-19 20:36:42 on dist-test-slave-0t40"
server_key: "c595b321322c6dc490870e1aecd6c263"
server_key_iv: "850618e7c488f523369a2f48ad975a83"
server_key_version: "encryptionkey@0"
I20260619 20:36:42.908574 31509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:36:42.909242 31524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:42.909456 31509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:36:42.909544 31509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "a665f65a8d73405d8803aefbb2a7472c"
format_stamp: "Formatted at 2026-06-19 20:36:42 on dist-test-slave-0t40"
server_key: "c595b321322c6dc490870e1aecd6c263"
server_key_iv: "850618e7c488f523369a2f48ad975a83"
server_key_version: "encryptionkey@0"
I20260619 20:36:42.909597 31509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:42.931644 31509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:42.933718 31509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:42.933831 31509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:42.934018 31509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:42.934332 31509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:42.934366 31509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:42.934387 31509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:42.934442 31509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:42.940274 31509 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:40855
I20260619 20:36:42.940454 31637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:40855 every 8 connection(s)
I20260619 20:36:42.940671 31509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:36:42.943859 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 31509
I20260619 20:36:42.943994 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:36:42.944192 25850 external_mini_cluster.cc:1428] Setting key efbf990b180647eebaad2430c6fce849
Jun 19 20:36:42 dist-test-slave-0t40 krb5kdc[31402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901402, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:42.951171 31487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:42.941412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:40787 (local address 127.25.62.190:35175)
0619 20:36:42.941490 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:42.941492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:42.941648 (+ 156us) server_negotiation.cc:407] Connection header received
0619 20:36:42.942538 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:42.942539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:42.942552 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:42.942609 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:42.943170 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:42.943660 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:42.945139 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:42.945369 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:42.945442 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:42.947858 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:42.947860 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:42.947860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:42.947871 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:42.949719 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:42.950148 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:42.950149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:42.950150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:42.950216 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:42.950453 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:42.950455 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:42.950455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:42.950601 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:42.950675 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:42.950968 (+ 293us) server_negotiation.cc:299] Negotiation successful
0619 20:36:42.951066 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 20:36:42.951189 31640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:42.941378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:35175 (local address 127.25.62.129:40787)
0619 20:36:42.941524 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:42.941564 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:42.942451 (+ 887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:42.942660 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:42.942664 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:42.942902 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:42.943097 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:42.943101 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:42.943749 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:42.943750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:42.945004 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:42.945008 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:42.945469 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:42.945470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:42.945620 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:42.946164 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:42.946175 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:42.947770 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:42.949818 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:42.949824 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:42.949825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:42.950063 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:42.950287 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:42.950287 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:42.950288 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:42.950388 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:42.950669 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:42.950673 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:42.950784 (+ 111us) client_negotiation.cc:769] Sending connection context
0619 20:36:42.950865 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 20:36:42.951035 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
I20260619 20:36:42.951821 31638 heartbeater.cc:347] Connected to a master server at 127.25.62.190:35175
I20260619 20:36:42.951908 31638 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:42.952106 31638 heartbeater.cc:511] Master 127.25.62.190:35175 requested a full tablet report, sending...
I20260619 20:36:42.952692 31447 ts_manager.cc:194] Registered new tserver with Master: a665f65a8d73405d8803aefbb2a7472c (127.25.62.129:40855)
WARNING: no policy specified for kudu/127.25.62.130@KRBTEST.COM; defaulting to no policy
I20260619 20:36:42.953624 31447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:40787
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
I20260619 20:36:42.971851 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:35175
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:43.050894 31643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:43.051093 31643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:43.051116 31643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:43.052201 31643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:43.052237 31643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:43.052263 31643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:36:43.053681 31643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:35175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:43.053855 31643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:43.054018 31643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:43.058848 31648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:43.058964 31643 server_base.cc:1034] running on GCE node
W20260619 20:36:43.058842 31649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:43.058838 31651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:43.059278 31643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:43.059540 31643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:43.060706 31643 hybrid_clock.cc:648] HybridClock initialized: now 1781901403060701 us; error 35 us; skew 500 ppm
Jun 19 20:36:43 dist-test-slave-0t40 krb5kdc[31402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901403, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:43.063191 31643 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:36:43.063606 31643 webserver.cc:460] Webserver started at http://127.25.62.130:44591/ using document root <none> and password file <none>
I20260619 20:36:43.063820 31643 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:43.063884 31643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:43.064028 31643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:43.065034 31643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "409a91c43b754288ac33301849a4f646"
format_stamp: "Formatted at 2026-06-19 20:36:43 on dist-test-slave-0t40"
server_key: "933be579312ec4c3aeadf34bccd73604"
server_key_iv: "ed447348fc8090a81f2f18edd983d8f4"
server_key_version: "encryptionkey@0"
I20260619 20:36:43.065459 31643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "409a91c43b754288ac33301849a4f646"
format_stamp: "Formatted at 2026-06-19 20:36:43 on dist-test-slave-0t40"
server_key: "933be579312ec4c3aeadf34bccd73604"
server_key_iv: "ed447348fc8090a81f2f18edd983d8f4"
server_key_version: "encryptionkey@0"
I20260619 20:36:43.066802 31643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:43.067497 31658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:43.067657 31643 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:43.067754 31643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "409a91c43b754288ac33301849a4f646"
format_stamp: "Formatted at 2026-06-19 20:36:43 on dist-test-slave-0t40"
server_key: "933be579312ec4c3aeadf34bccd73604"
server_key_iv: "ed447348fc8090a81f2f18edd983d8f4"
server_key_version: "encryptionkey@0"
I20260619 20:36:43.067832 31643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:43.081664 31643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:43.083474 31643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:43.083632 31643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:43.083845 31643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:43.084157 31643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:43.084213 31643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:43.084254 31643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:43.084278 31643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:43.090114 31643 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:37605
I20260619 20:36:43.090163 31771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:37605 every 8 connection(s)
I20260619 20:36:43.090458 31643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:36:43 dist-test-slave-0t40 krb5kdc[31402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901403, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:43.096484 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 31643
I20260619 20:36:43.096611 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:36:43.096822 25850 external_mini_cluster.cc:1428] Setting key b911cf531b04eee98487d961e6fd1c2e
I20260619 20:36:43.100414 31487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:43.091101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:50905 (local address 127.25.62.190:35175)
0619 20:36:43.091166 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:43.091168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:43.091339 (+ 171us) server_negotiation.cc:407] Connection header received
0619 20:36:43.092234 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:43.092235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:43.092246 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:43.092307 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:43.092878 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:43.093437 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:43.094413 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:43.094625 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:43.094699 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:43.097121 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:43.097122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:43.097123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:43.097131 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:43.098862 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:43.099328 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:43.099330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:43.099330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:43.099417 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:43.099693 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:43.099694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:43.099694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:43.099828 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:43.099918 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:43.100201 (+ 283us) server_negotiation.cc:299] Negotiation successful
0619 20:36:43.100304 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:36:43.100575 31774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:43.091055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:35175 (local address 127.25.62.130:50905)
0619 20:36:43.091225 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:43.091267 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:43.092146 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:43.092385 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:43.092389 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:43.092630 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:43.092808 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:43.092812 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:43.093523 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:43.093523 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:43.094306 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:43.094308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:43.094729 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:43.094730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:43.094839 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:43.095368 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:43.095376 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:43.096967 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:43.098954 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:43.098960 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:43.098962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:43.099211 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:43.099491 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:43.099492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:43.099492 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:43.099615 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:43.099906 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:43.099912 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:43.100061 (+ 149us) client_negotiation.cc:769] Sending connection context
0619 20:36:43.100179 (+ 118us) client_negotiation.cc:240] Negotiation successful
0619 20:36:43.100339 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260619 20:36:43.101181 31772 heartbeater.cc:347] Connected to a master server at 127.25.62.190:35175
I20260619 20:36:43.101295 31772 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:43.101490 31772 heartbeater.cc:511] Master 127.25.62.190:35175 requested a full tablet report, sending...
I20260619 20:36:43.101889 31447 ts_manager.cc:194] Registered new tserver with Master: 409a91c43b754288ac33301849a4f646 (127.25.62.130:37605)
I20260619 20:36:43.102270 31447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:50905
WARNING: no policy specified for kudu/127.25.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
I20260619 20:36:43.124408 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:35175
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:43.203794 31777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:43.204030 31777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:43.204067 31777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:43.205269 31777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:43.205327 31777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:43.205370 31777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:36:43.206863 31777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:35175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:43.207131 31777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:43.207351 31777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:43.212258 31785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:43.212379 31777 server_base.cc:1034] running on GCE node
W20260619 20:36:43.212240 31783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:43.212203 31782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:43.212708 31777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:43.212951 31777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:43.214085 31777 hybrid_clock.cc:648] HybridClock initialized: now 1781901403214076 us; error 48 us; skew 500 ppm
Jun 19 20:36:43 dist-test-slave-0t40 krb5kdc[31402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901403, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:43.216480 31777 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:36:43.216885 31777 webserver.cc:460] Webserver started at http://127.25.62.131:46771/ using document root <none> and password file <none>
I20260619 20:36:43.217155 31777 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:43.217222 31777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:43.217377 31777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:43.218485 31777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "91f20759fd5142abae2e345cac260278"
format_stamp: "Formatted at 2026-06-19 20:36:43 on dist-test-slave-0t40"
server_key: "ef052e174bbbba8694264ba275aad73c"
server_key_iv: "45c56fbf942cd40e8259c1934ec05f5f"
server_key_version: "encryptionkey@0"
I20260619 20:36:43.218847 31777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "91f20759fd5142abae2e345cac260278"
format_stamp: "Formatted at 2026-06-19 20:36:43 on dist-test-slave-0t40"
server_key: "ef052e174bbbba8694264ba275aad73c"
server_key_iv: "45c56fbf942cd40e8259c1934ec05f5f"
server_key_version: "encryptionkey@0"
I20260619 20:36:43.220141 31777 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:36:43.220875 31792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:43.221103 31777 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:43.221206 31777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "91f20759fd5142abae2e345cac260278"
format_stamp: "Formatted at 2026-06-19 20:36:43 on dist-test-slave-0t40"
server_key: "ef052e174bbbba8694264ba275aad73c"
server_key_iv: "45c56fbf942cd40e8259c1934ec05f5f"
server_key_version: "encryptionkey@0"
I20260619 20:36:43.221292 31777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:43.231541 31777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:43.233640 31777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:43.233810 31777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:43.234040 31777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:43.234365 31777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:43.234405 31777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:43.234428 31777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:43.234440 31777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:43.239976 31777 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:35271
I20260619 20:36:43.240046 31905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:35271 every 8 connection(s)
I20260619 20:36:43.240388 31777 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:36:43 dist-test-slave-0t40 krb5kdc[31402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901403, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:43.248950 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 31777
I20260619 20:36:43.249071 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:36:43.249279 25850 external_mini_cluster.cc:1428] Setting key c52f043d619190acbe0c61885f80fd16
I20260619 20:36:43.250304 31487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:43.241210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:60245 (local address 127.25.62.190:35175)
0619 20:36:43.241296 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:43.241298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:43.241495 (+ 197us) server_negotiation.cc:407] Connection header received
0619 20:36:43.242427 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:43.242428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:43.242442 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:43.242504 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:43.243151 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:43.243660 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:43.244678 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:43.244873 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:43.244957 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:43.247383 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:43.247385 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:43.247385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:43.247393 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:43.248703 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:43.249215 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:43.249216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:43.249216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:43.249278 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:43.249598 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:43.249599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:43.249599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:43.249749 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:43.249823 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:43.250137 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 20:36:43.250216 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 20:36:43.250485 31908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:43.241167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:35175 (local address 127.25.62.131:60245)
0619 20:36:43.241338 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:43.241379 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:43.242309 (+ 930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:43.242573 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:43.242577 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:43.242847 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:43.243046 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:43.243050 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:43.243760 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:43.243761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:43.244573 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:43.244576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:43.244974 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:43.244975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:43.245160 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:43.245639 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:43.245647 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:43.247230 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:43.248799 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:43.248805 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:43.248807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:43.249095 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:43.249353 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:43.249354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:43.249354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:43.249500 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:43.249819 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:43.249824 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:43.249997 (+ 173us) client_negotiation.cc:769] Sending connection context
0619 20:36:43.250098 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 20:36:43.250252 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260619 20:36:43.250872 31906 heartbeater.cc:347] Connected to a master server at 127.25.62.190:35175
I20260619 20:36:43.250965 31906 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:43.251102 31906 heartbeater.cc:511] Master 127.25.62.190:35175 requested a full tablet report, sending...
I20260619 20:36:43.251410 31447 ts_manager.cc:194] Registered new tserver with Master: 91f20759fd5142abae2e345cac260278 (127.25.62.131:35271)
I20260619 20:36:43.251734 31447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:60245
I20260619 20:36:43.261137 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/test-admin.keytab.
Jun 19 20:36:43 dist-test-slave-0t40 krb5kdc[31402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:43.277396 25850 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 20:36:43.279008 31913 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:43.278223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41994 (local address 127.25.62.129:40855)
0619 20:36:43.278374 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:43.278384 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:43.278395 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:36:43.278421 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:43.278423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:43.278437 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:43.278506 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:43.278895 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41994: BlockingRecv error: recv got EOF from 127.0.0.1:41994 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
W20260619 20:36:43.279194 25850 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/krb5cc' of type FILE
Jun 19 20:36:43 dist-test-slave-0t40 krb5kdc[31402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:43.280159 25850 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 20:36:43 dist-test-slave-0t40 krb5kdc[31402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:43.280988 25850 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 20:36:43 dist-test-slave-0t40 krb5kdc[31402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.129@KRBTEST.COM
I20260619 20:36:43.288681 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:43.281494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42006 (local address 127.25.62.129:40855)
0619 20:36:43.281556 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:43.281559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:43.281618 (+ 59us) server_negotiation.cc:407] Connection header received
0619 20:36:43.281694 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:43.281695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:43.281706 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:43.281777 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:43.282187 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:43.282918 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:43.283655 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:43.283985 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:43.284076 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:43.286174 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:43.286181 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:43.286182 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:43.286189 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:43.287547 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:43.287903 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:43.287906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:43.287906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:43.287964 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:43.288161 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:43.288162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:43.288163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:43.288292 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:43.288437 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:43.288494 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 20:36:43.288572 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260619 20:36:43.290355 31913 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:43.289880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42016 (local address 127.25.62.129:40855)
0619 20:36:43.289912 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:43.289914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:43.289958 (+ 44us) server_negotiation.cc:407] Connection header received
0619 20:36:43.290010 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:43.290011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:43.290019 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:43.290069 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:43.290309 (+ 240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42016: BlockingRecv error: recv got EOF from 127.0.0.1:42016 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 20:36:43.290452 25850 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/krb5cc' of type FILE
Jun 19 20:36:43 dist-test-slave-0t40 krb5kdc[31402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:43.291314 25850 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 20:36:43 dist-test-slave-0t40 krb5kdc[31402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:43.292207 25850 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 20:36:43 dist-test-slave-0t40 krb5kdc[31402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.129@KRBTEST.COM
I20260619 20:36:43.298554 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:43.292695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42024 (local address 127.25.62.129:40855)
0619 20:36:43.292761 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:43.292763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:43.292771 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:43.292821 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:43.292821 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:43.292828 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:43.292868 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:43.293268 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:43.293724 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:43.294392 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:43.294606 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:43.294679 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:43.296485 (+ 1806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:43.296486 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:43.296487 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:43.296492 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:43.297685 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:43.297966 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:43.297966 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:43.297967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:43.298012 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:43.298152 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:43.298152 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:43.298152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:43.298221 (+ 69us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:43.298281 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:43.298360 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 20:36:43.298420 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260619 20:36:43.299980 31913 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:43.299488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42030 (local address 127.25.62.129:40855)
0619 20:36:43.299548 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:43.299549 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:43.299559 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:36:43.299618 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:43.299619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:43.299628 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:43.299684 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:43.299940 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42030: BlockingRecv error: recv got EOF from 127.0.0.1:42030 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260619 20:36:43.300091 25850 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781901378226483-25850-0/krb5kdc/krb5cc' of type FILE
Jun 19 20:36:43 dist-test-slave-0t40 krb5kdc[31402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:43.300953 25850 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 20:36:43 dist-test-slave-0t40 krb5kdc[31402](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:43.301774 25850 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 20:36:43 dist-test-slave-0t40 krb5kdc[31402](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.129@KRBTEST.COM
I20260619 20:36:43.307938 31913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:43.302240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42040 (local address 127.25.62.129:40855)
0619 20:36:43.302273 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:43.302275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:43.302306 (+ 31us) server_negotiation.cc:407] Connection header received
0619 20:36:43.302348 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:43.302349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:43.302355 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:43.302398 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:43.302693 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:43.303105 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:43.303699 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:43.303865 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:43.303912 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:43.305755 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:43.305756 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:43.305756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:43.305762 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:43.306993 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:43.307324 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:43.307325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:43.307325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:43.307374 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:43.307533 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:43.307533 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:43.307533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:43.307610 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:43.307671 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:43.307751 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 20:36:43.307801 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:36:43.308612 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 31509
I20260619 20:36:43.313228 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 31643
I20260619 20:36:43.317490 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 31777
I20260619 20:36:43.321393 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 31415
2026-06-19T20:36:43Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1863 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:43 dist-test-slave-0t40 krb5kdc[31926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:43 dist-test-slave-0t40 krb5kdc[31926](info): set up 2 sockets
Jun 19 20:36:43 dist-test-slave-0t40 krb5kdc[31926](info): commencing operation
krb5kdc: starting...
W20260619 20:36:44.353454 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:36:44 dist-test-slave-0t40 krb5kdc[31926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:44Z Disabled control of system clock
WARNING: no policy specified for oryx/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.190.keytab.
Entry for principal oryx/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.190.keytab.
I20260619 20:36:44.471688 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:42409
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.190.keytab
--principal=oryx/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:45475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:42409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:44.550164 31939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:44.550393 31939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:44.550427 31939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:44.551541 31939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:44.551607 31939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:44.551632 31939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:44.551661 31939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:44.551694 31939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:44.553148 31939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:45475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:42409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.190.keytab
--principal=oryx/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:42409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:44.553411 31939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:44.553629 31939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:44.558329 31944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:44.558341 31945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:44.558351 31947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:44.558561 31939 server_base.cc:1034] running on GCE node
I20260619 20:36:44.558799 31939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:44.559087 31939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:44.560230 31939 hybrid_clock.cc:648] HybridClock initialized: now 1781901404560211 us; error 40 us; skew 500 ppm
Jun 19 20:36:44 dist-test-slave-0t40 krb5kdc[31926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901404, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:44.565996 31939 init.cc:377] Logged in from keytab as oryx/127.25.62.190@KRBTEST.COM (short username oryx)
I20260619 20:36:44.566426 31939 webserver.cc:460] Webserver started at http://127.25.62.190:40705/ using document root <none> and password file <none>
I20260619 20:36:44.566655 31939 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:44.566723 31939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:44.566872 31939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:44.567924 31939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "d46f1eb7b6e34656a7113c47eafa65e4"
format_stamp: "Formatted at 2026-06-19 20:36:44 on dist-test-slave-0t40"
server_key: "615dba4097fe222ab5a9916ebb36ecbf"
server_key_iv: "d50b8955ed271f592ccce8d767463c7b"
server_key_version: "encryptionkey@0"
I20260619 20:36:44.568287 31939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "d46f1eb7b6e34656a7113c47eafa65e4"
format_stamp: "Formatted at 2026-06-19 20:36:44 on dist-test-slave-0t40"
server_key: "615dba4097fe222ab5a9916ebb36ecbf"
server_key_iv: "d50b8955ed271f592ccce8d767463c7b"
server_key_version: "encryptionkey@0"
I20260619 20:36:44.569679 31939 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:36:44.570363 31954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:44.570541 31939 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:44.570645 31939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "d46f1eb7b6e34656a7113c47eafa65e4"
format_stamp: "Formatted at 2026-06-19 20:36:44 on dist-test-slave-0t40"
server_key: "615dba4097fe222ab5a9916ebb36ecbf"
server_key_iv: "d50b8955ed271f592ccce8d767463c7b"
server_key_version: "encryptionkey@0"
I20260619 20:36:44.570719 31939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:44.587962 31939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:44.590057 31939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:44.590227 31939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:44.594239 31939 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:42409
I20260619 20:36:44.594274 32006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:42409 every 8 connection(s)
I20260619 20:36:44.594641 31939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:44.595608 32007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:44.596371 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 31939
I20260619 20:36:44.596498 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:36:44.596683 25850 external_mini_cluster.cc:1428] Setting key 4b77906abdd408009f83bb44911cc695
I20260619 20:36:44.598726 32007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4: Bootstrap starting.
I20260619 20:36:44.599339 32007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:44.599642 32007 log.cc:824] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:44.600376 32007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4: No bootstrap required, opened a new log
I20260619 20:36:44.602361 32007 raft_consensus.cc:348] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } }
I20260619 20:36:44.602515 32007 raft_consensus.cc:374] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:44.602560 32007 raft_consensus.cc:729] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d46f1eb7b6e34656a7113c47eafa65e4, State: Initialized, Role: FOLLOWER
I20260619 20:36:44.602687 32007 consensus_queue.cc:260] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } }
I20260619 20:36:44.602792 32007 raft_consensus.cc:388] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:44.602835 32007 raft_consensus.cc:482] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:44.602882 32007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [term 0 FOLLOWER]: Advancing to term 1
Jun 19 20:36:44 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM
I20260619 20:36:44.603556 32007 raft_consensus.cc:504] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } }
I20260619 20:36:44.603653 32007 leader_election.cc:302] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d46f1eb7b6e34656a7113c47eafa65e4; no voters:
I20260619 20:36:44.603835 32007 leader_election.cc:288] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:44.603869 32012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:44.603978 32012 raft_consensus.cc:686] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [term 1 LEADER]: Becoming Leader. State: Replica: d46f1eb7b6e34656a7113c47eafa65e4, State: Running, Role: LEADER
I20260619 20:36:44.604064 32012 consensus_queue.cc:237] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } }
I20260619 20:36:44.604096 32007 sys_catalog.cc:564] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:44.604367 32012 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } }
I20260619 20:36:44.604488 32012 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:44.604625 32012 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d46f1eb7b6e34656a7113c47eafa65e4. Latest consensus state: current_term: 1 leader_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } }
I20260619 20:36:44.604763 32012 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:44.605193 32018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:44.605748 32018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:44.607569 32018 catalog_manager.cc:1347] Generated new cluster ID: e87fcace73ff40c79a43b072c099bda4
I20260619 20:36:44.607627 32018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:44.608325 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:44.597247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51388 (local address 127.25.62.190:42409)
0619 20:36:44.597416 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:44.597432 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:44.597457 (+ 25us) server_negotiation.cc:407] Connection header received
0619 20:36:44.597842 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:44.597848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:44.597892 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:44.598010 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:44.598774 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:44.599775 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:44.600765 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:44.601248 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:44.601393 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:44.604239 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:44.604243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:44.604244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:44.604253 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:44.606608 (+ 2355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:44.607044 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:44.607047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:44.607047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:44.607152 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:44.607378 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:44.607379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:44.607379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:44.607688 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:44.607825 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:44.607989 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 20:36:44.608128 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260619 20:36:44.617642 32018 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:44.618335 32018 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:44.623122 32018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4: Generated new TSK 0
I20260619 20:36:44.623286 32018 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.129.keytab.
Entry for principal oryx/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.129.keytab.
I20260619 20:36:44.653585 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:42409
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.129.keytab
--principal=oryx/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:45475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:44.736625 32033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:44.736856 32033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:44.736897 32033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:44.738145 32033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:44.738215 32033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:44.738263 32033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:36:44.739804 32033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:45475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.129.keytab
--principal=oryx/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:42409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:44.740072 32033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:44.740302 32033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:44.745407 32039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:44.745462 32038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:44.745450 32041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:44.745517 32033 server_base.cc:1034] running on GCE node
I20260619 20:36:44.745765 32033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:44.746006 32033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:44.747179 32033 hybrid_clock.cc:648] HybridClock initialized: now 1781901404747172 us; error 35 us; skew 500 ppm
Jun 19 20:36:44 dist-test-slave-0t40 krb5kdc[31926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901404, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:44.749755 32033 init.cc:377] Logged in from keytab as oryx/127.25.62.129@KRBTEST.COM (short username oryx)
I20260619 20:36:44.750174 32033 webserver.cc:460] Webserver started at http://127.25.62.129:34757/ using document root <none> and password file <none>
I20260619 20:36:44.750411 32033 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:44.750478 32033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:44.750635 32033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:44.751672 32033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "afca6bfafe0342f6a6e7167ae9632164"
format_stamp: "Formatted at 2026-06-19 20:36:44 on dist-test-slave-0t40"
server_key: "4c37c76e82bee9c058e1ab96557ba4f4"
server_key_iv: "e88559d44ecd6e4d8dbd1c20294cc7a3"
server_key_version: "encryptionkey@0"
I20260619 20:36:44.752035 32033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "afca6bfafe0342f6a6e7167ae9632164"
format_stamp: "Formatted at 2026-06-19 20:36:44 on dist-test-slave-0t40"
server_key: "4c37c76e82bee9c058e1ab96557ba4f4"
server_key_iv: "e88559d44ecd6e4d8dbd1c20294cc7a3"
server_key_version: "encryptionkey@0"
I20260619 20:36:44.753366 32033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:36:44.754052 32048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:44.754218 32033 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:36:44.754307 32033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "afca6bfafe0342f6a6e7167ae9632164"
format_stamp: "Formatted at 2026-06-19 20:36:44 on dist-test-slave-0t40"
server_key: "4c37c76e82bee9c058e1ab96557ba4f4"
server_key_iv: "e88559d44ecd6e4d8dbd1c20294cc7a3"
server_key_version: "encryptionkey@0"
I20260619 20:36:44.754386 32033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:44.771193 32033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:44.773595 32033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:44.773775 32033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:44.774024 32033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:44.774330 32033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:44.774385 32033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:44.774417 32033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:44.774437 32033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:44.780073 32033 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:33507
I20260619 20:36:44.780157 32161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:33507 every 8 connection(s)
I20260619 20:36:44.780470 32033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:36:44 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901404, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM
I20260619 20:36:44.788421 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 32033
I20260619 20:36:44.788537 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:36:44.788738 25850 external_mini_cluster.cc:1428] Setting key 661ded44a894c3ea72cb81bc7f518ede
I20260619 20:36:44.790534 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:44.781222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:59547 (local address 127.25.62.190:42409)
0619 20:36:44.781266 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:44.781267 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:44.781493 (+ 226us) server_negotiation.cc:407] Connection header received
0619 20:36:44.782025 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:44.782027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:44.782039 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:44.782099 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:44.783130 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:44.783644 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:44.784646 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:44.784875 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:44.784953 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:44.787446 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:44.787447 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:44.787447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:44.787455 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:44.789091 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:44.789570 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:44.789571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:44.789572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:44.789632 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:44.789894 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:44.789895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:44.789896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:44.790012 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:44.790088 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:44.790355 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 20:36:44.790443 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:36:44.790845 32164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:44.781212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.129:59547)
0619 20:36:44.781356 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:44.781401 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:44.781916 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:44.782171 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:44.782174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:44.782833 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:44.783024 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:44.783028 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:44.783740 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:44.783741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:44.784541 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:44.784543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:44.784972 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:44.784973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:44.785107 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:44.785666 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:44.785679 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:44.787325 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:44.789167 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:44.789173 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:44.789175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:44.789485 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:44.789705 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:44.789706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:44.789706 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:44.789819 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:44.790078 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:44.790085 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:44.790222 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 20:36:44.790323 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 20:36:44.790484 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260619 20:36:44.791281 32162 heartbeater.cc:347] Connected to a master server at 127.25.62.190:42409
I20260619 20:36:44.791368 32162 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:44.791563 32162 heartbeater.cc:511] Master 127.25.62.190:42409 requested a full tablet report, sending...
I20260619 20:36:44.792060 31971 ts_manager.cc:194] Registered new tserver with Master: afca6bfafe0342f6a6e7167ae9632164 (127.25.62.129:33507)
I20260619 20:36:44.792649 31971 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:59547
WARNING: no policy specified for oryx/127.25.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.130.keytab.
Entry for principal oryx/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.130.keytab.
I20260619 20:36:44.815790 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:42409
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.130.keytab
--principal=oryx/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:45475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:44.894470 32167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:44.894667 32167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:44.894690 32167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:44.895792 32167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:44.895829 32167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:44.895854 32167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:36:44.897284 32167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:45475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.130.keytab
--principal=oryx/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:42409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:44.897519 32167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:44.897753 32167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:44.902612 32173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:44.902585 32175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:44.902585 32172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:44.902897 32167 server_base.cc:1034] running on GCE node
I20260619 20:36:44.903055 32167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:44.903230 32167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:44.904359 32167 hybrid_clock.cc:648] HybridClock initialized: now 1781901404904346 us; error 30 us; skew 500 ppm
Jun 19 20:36:44 dist-test-slave-0t40 krb5kdc[31926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901404, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:44.906908 32167 init.cc:377] Logged in from keytab as oryx/127.25.62.130@KRBTEST.COM (short username oryx)
I20260619 20:36:44.907262 32167 webserver.cc:460] Webserver started at http://127.25.62.130:36401/ using document root <none> and password file <none>
I20260619 20:36:44.907440 32167 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:44.907480 32167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:44.907565 32167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:44.908506 32167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "a576e1317b724daeaea1183f19e0548e"
format_stamp: "Formatted at 2026-06-19 20:36:44 on dist-test-slave-0t40"
server_key: "216045aa00d9ca23c3e3ba7036278806"
server_key_iv: "e7aa066eb8a391b1e11ae571f93c13ea"
server_key_version: "encryptionkey@0"
I20260619 20:36:44.908782 32167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "a576e1317b724daeaea1183f19e0548e"
format_stamp: "Formatted at 2026-06-19 20:36:44 on dist-test-slave-0t40"
server_key: "216045aa00d9ca23c3e3ba7036278806"
server_key_iv: "e7aa066eb8a391b1e11ae571f93c13ea"
server_key_version: "encryptionkey@0"
I20260619 20:36:44.910053 32167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:36:44.910694 32182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:44.910861 32167 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:36:44.910933 32167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "a576e1317b724daeaea1183f19e0548e"
format_stamp: "Formatted at 2026-06-19 20:36:44 on dist-test-slave-0t40"
server_key: "216045aa00d9ca23c3e3ba7036278806"
server_key_iv: "e7aa066eb8a391b1e11ae571f93c13ea"
server_key_version: "encryptionkey@0"
I20260619 20:36:44.911020 32167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:44.925225 32167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:44.927453 32167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:44.927613 32167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:44.927860 32167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:44.928179 32167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:44.928217 32167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:44.928239 32167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:44.928287 32167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:44.933903 32167 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:36761
I20260619 20:36:44.933951 32295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:36761 every 8 connection(s)
I20260619 20:36:44.934235 32167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:36:44 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901404, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM
I20260619 20:36:44.940454 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 32167
I20260619 20:36:44.940563 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:36:44.940773 25850 external_mini_cluster.cc:1428] Setting key 0b4a6f802af3e009e9c9905a1c0da22c
I20260619 20:36:44.943466 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:44.934957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:44725 (local address 127.25.62.190:42409)
0619 20:36:44.935036 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:44.935038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:44.935200 (+ 162us) server_negotiation.cc:407] Connection header received
0619 20:36:44.935641 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:44.935642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:44.935652 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:44.935699 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:44.936558 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:44.937077 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:44.938026 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:44.938208 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:44.938262 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:44.940371 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:44.940372 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:44.940373 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:44.940381 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:44.942074 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:44.942481 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:44.942482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:44.942483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:44.942544 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:44.942767 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:44.942768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:44.942769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:44.942904 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:44.942978 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:44.943338 (+ 360us) server_negotiation.cc:299] Negotiation successful
0619 20:36:44.943400 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 20:36:44.943753 32298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:44.934888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.130:44725)
0619 20:36:44.935072 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:44.935106 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:44.935559 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:44.935759 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:44.935761 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:44.936287 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:44.936480 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:44.936486 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:44.937166 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:44.937167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:44.937951 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:44.937953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:44.938303 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:44.938304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:44.938421 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:44.938958 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:44.938966 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:44.940271 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:44.942174 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:44.942180 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:44.942181 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:44.942401 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:44.942610 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:44.942611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:44.942611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:44.942706 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:44.943000 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:44.943008 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:44.943190 (+ 182us) client_negotiation.cc:769] Sending connection context
0619 20:36:44.943316 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 20:36:44.943506 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":101,"threads_started":1}
I20260619 20:36:44.944236 32296 heartbeater.cc:347] Connected to a master server at 127.25.62.190:42409
I20260619 20:36:44.944324 32296 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:44.944509 32296 heartbeater.cc:511] Master 127.25.62.190:42409 requested a full tablet report, sending...
I20260619 20:36:44.944855 31971 ts_manager.cc:194] Registered new tserver with Master: a576e1317b724daeaea1183f19e0548e (127.25.62.130:36761)
I20260619 20:36:44.945344 31971 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:44725
WARNING: no policy specified for oryx/127.25.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.131.keytab.
Entry for principal oryx/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.131.keytab.
I20260619 20:36:44.966287 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:42409
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.131.keytab
--principal=oryx/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:45475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:45.046872 32301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:45.047106 32301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:45.047145 32301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:45.048250 32301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:45.048322 32301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:45.048367 32301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:36:45.049782 32301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:45475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.131.keytab
--principal=oryx/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:42409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:45.050042 32301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:45.050287 32301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:45.055229 32307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:45.055418 32301 server_base.cc:1034] running on GCE node
W20260619 20:36:45.055207 32309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:45.055207 32306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:45.055681 32301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:45.055953 32301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:45.057097 32301 hybrid_clock.cc:648] HybridClock initialized: now 1781901405057084 us; error 38 us; skew 500 ppm
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901405, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:45.059707 32301 init.cc:377] Logged in from keytab as oryx/127.25.62.131@KRBTEST.COM (short username oryx)
I20260619 20:36:45.060191 32301 webserver.cc:460] Webserver started at http://127.25.62.131:40419/ using document root <none> and password file <none>
I20260619 20:36:45.060437 32301 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:45.060508 32301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:45.060657 32301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:45.061817 32301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "7589a1cd5a8d467cbd7d9610a2db3936"
format_stamp: "Formatted at 2026-06-19 20:36:45 on dist-test-slave-0t40"
server_key: "6b7a16daaac7050ebdf820ab21b2ca3d"
server_key_iv: "5151a54270942315fd139b3d29603fb4"
server_key_version: "encryptionkey@0"
I20260619 20:36:45.062180 32301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "7589a1cd5a8d467cbd7d9610a2db3936"
format_stamp: "Formatted at 2026-06-19 20:36:45 on dist-test-slave-0t40"
server_key: "6b7a16daaac7050ebdf820ab21b2ca3d"
server_key_iv: "5151a54270942315fd139b3d29603fb4"
server_key_version: "encryptionkey@0"
I20260619 20:36:45.063438 32301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:36:45.064170 32316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:45.064354 32301 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:36:45.064459 32301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "7589a1cd5a8d467cbd7d9610a2db3936"
format_stamp: "Formatted at 2026-06-19 20:36:45 on dist-test-slave-0t40"
server_key: "6b7a16daaac7050ebdf820ab21b2ca3d"
server_key_iv: "5151a54270942315fd139b3d29603fb4"
server_key_version: "encryptionkey@0"
I20260619 20:36:45.064533 32301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:45.093524 32301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:45.095816 32301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:45.096011 32301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:45.096251 32301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:45.096578 32301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:45.096634 32301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:45.096676 32301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:45.096705 32301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:45.102628 32301 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:43649
I20260619 20:36:45.102670 32429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:43649 every 8 connection(s)
I20260619 20:36:45.102986 32301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901405, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM
I20260619 20:36:45.111093 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 32301
I20260619 20:36:45.111196 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:36:45.111388 25850 external_mini_cluster.cc:1428] Setting key 41503cf080ed2f2497d20a810b98e017
I20260619 20:36:45.112746 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.103711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:56381 (local address 127.25.62.190:42409)
0619 20:36:45.103760 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.103762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.104042 (+ 280us) server_negotiation.cc:407] Connection header received
0619 20:36:45.104634 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.104636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.104649 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.104702 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.105794 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.106296 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.107302 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.107545 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.107615 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.109888 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.109890 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.109890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.109898 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.111352 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.111839 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.111840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.111841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.111906 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.112152 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.112153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.112153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.112296 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.112365 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.112616 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.112683 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:36:45.112995 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.103710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.131:56381)
0619 20:36:45.103884 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.103947 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.104513 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.104770 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.104774 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.105467 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.105698 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.105703 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.106387 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.106388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.107180 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.107182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.107621 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.107623 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.107744 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.108270 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.108280 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.109787 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.111469 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.111476 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.111478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.111764 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.111976 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.111977 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.111977 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.112093 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.112370 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.112376 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.112505 (+ 129us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.112589 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.112734 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260619 20:36:45.113449 32430 heartbeater.cc:347] Connected to a master server at 127.25.62.190:42409
I20260619 20:36:45.113575 32430 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:45.113751 32430 heartbeater.cc:511] Master 127.25.62.190:42409 requested a full tablet report, sending...
I20260619 20:36:45.114128 31971 ts_manager.cc:194] Registered new tserver with Master: 7589a1cd5a8d467cbd7d9610a2db3936 (127.25.62.131:43649)
I20260619 20:36:45.114499 31971 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:56381
I20260619 20:36:45.123162 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.25.62.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.62.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.62.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.189.keytab.
Entry for principal oryx/127.25.62.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.189.keytab.
I20260619 20:36:45.147593 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.189:36729
--webserver_interface=127.25.62.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.189.keytab
--principal=oryx/127.25.62.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:45475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:42409,127.25.62.189:36729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:45.227150 32435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:45.227335 32435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:45.227355 32435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:45.228456 32435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:45.228495 32435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:45.228509 32435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:45.228545 32435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:45.228578 32435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:45.230024 32435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:45475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:42409,127.25.62.189:36729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.189.keytab
--principal=oryx/127.25.62.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.189:36729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.62.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
W20260619 20:36:45.230121 32435 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.25.62.190:42409,127.25.62.189:36729'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 20:36:45.230223 32435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:45.230372 32435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:36:45.235110 32435 server_base.cc:1034] running on GCE node
W20260619 20:36:45.235077 32441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:45.235080 32440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:45.235073 32443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:45.235587 32435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:45.235879 32435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:45.237107 32435 hybrid_clock.cc:648] HybridClock initialized: now 1781901405237084 us; error 41 us; skew 500 ppm
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901405, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:45.239517 32435 init.cc:377] Logged in from keytab as oryx/127.25.62.189@KRBTEST.COM (short username oryx)
I20260619 20:36:45.239974 32435 webserver.cc:460] Webserver started at http://127.25.62.189:41333/ using document root <none> and password file <none>
I20260619 20:36:45.240193 32435 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:45.240236 32435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:45.240396 32435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:45.241492 32435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/data/instance:
uuid: "043ffdae4c624a6797578e3edeb62d28"
format_stamp: "Formatted at 2026-06-19 20:36:45 on dist-test-slave-0t40"
server_key: "9d0e8ad5049117e66569134c8948c4e8"
server_key_iv: "60dca38b6f11064e20bc6b44d0c2daf6"
server_key_version: "encryptionkey@0"
I20260619 20:36:45.241830 32435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/wal/instance:
uuid: "043ffdae4c624a6797578e3edeb62d28"
format_stamp: "Formatted at 2026-06-19 20:36:45 on dist-test-slave-0t40"
server_key: "9d0e8ad5049117e66569134c8948c4e8"
server_key_iv: "60dca38b6f11064e20bc6b44d0c2daf6"
server_key_version: "encryptionkey@0"
I20260619 20:36:45.243070 32435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:36:45.243724 32450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:45.243912 32435 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:45.243999 32435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/wal
uuid: "043ffdae4c624a6797578e3edeb62d28"
format_stamp: "Formatted at 2026-06-19 20:36:45 on dist-test-slave-0t40"
server_key: "9d0e8ad5049117e66569134c8948c4e8"
server_key_iv: "60dca38b6f11064e20bc6b44d0c2daf6"
server_key_version: "encryptionkey@0"
I20260619 20:36:45.244067 32435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:45.265002 32435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:45.267382 32435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:45.267530 32435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:45.271540 32435 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.189:36729
I20260619 20:36:45.271600 32502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.189:36729 every 8 connection(s)
I20260619 20:36:45.271929 32435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/data/info.pb
I20260619 20:36:45.272130 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 32435
I20260619 20:36:45.273145 32503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:45.275027 32503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.62.189@KRBTEST.COM
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901405, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.189@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM
I20260619 20:36:45.282727 32506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.272741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44050 (local address 127.25.62.189:36729)
0619 20:36:45.272960 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.272970 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.273000 (+ 30us) server_negotiation.cc:407] Connection header received
0619 20:36:45.273590 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.273599 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.273659 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.273807 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.274429 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.275728 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.276587 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.276974 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.277128 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.279567 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.279574 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.279575 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.279582 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.281182 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.281602 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.281604 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.281604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.281678 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.281876 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.281876 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.281876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.282073 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.282218 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.282315 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.282489 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":106,"threads_started":1}
I20260619 20:36:45.283386 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.275680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51404 (local address 127.25.62.190:42409)
0619 20:36:45.275744 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.275746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.275872 (+ 126us) server_negotiation.cc:407] Connection header received
0619 20:36:45.275936 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.275937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.275955 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.276015 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.276385 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.276923 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.277954 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.278155 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.278236 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.280633 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.280634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.280635 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.280641 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.282136 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.282564 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.282565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.282565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.282613 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.282807 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.282808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.282808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.282999 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.283053 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.283226 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.283305 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 20:36:45.283409 32507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.275643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.0.0.1:51404)
0619 20:36:45.275788 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.275826 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.275889 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.276034 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.276036 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.276252 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.276313 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.276315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.277060 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.277061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.277851 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.277853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.278247 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.278248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.278320 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.278764 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.278775 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.280551 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.282212 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.282216 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.282218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.282495 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.282670 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.282671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.282671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.282733 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.283049 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.283052 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.283097 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.283186 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.283303 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":93,"threads_started":1}
I20260619 20:36:45.283947 32503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901405, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.189@KRBTEST.COM for oryx/127.25.62.189@KRBTEST.COM
I20260619 20:36:45.289302 32507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.284171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:36729 (local address 127.0.0.1:44054)
0619 20:36:45.284217 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.284228 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.284266 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.284373 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.284374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.284480 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.284525 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.284526 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.285091 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.285092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.285559 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.285560 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.285861 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.285862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.285907 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.286277 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.286281 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.287329 (+ 1048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.288533 (+ 1204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.288534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.288534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.288679 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.288836 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.288837 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.288837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.288882 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.289084 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.289085 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.289129 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.289182 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.289249 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 20:36:45.289438 32506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.284223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44054 (local address 127.25.62.189:36729)
0619 20:36:45.284268 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.284270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.284278 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:45.284323 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.284324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.284331 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.284372 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.284565 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.284996 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.285619 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.285801 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.285849 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.287392 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.287393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.287393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.287399 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.288468 (+ 1069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.288737 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.288738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.288738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.288783 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.288924 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.288924 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.288924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.289003 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.289100 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.289198 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.289291 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":28}
I20260619 20:36:45.290683 32503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: Bootstrap starting.
I20260619 20:36:45.291529 32503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:45.291796 32503 log.cc:824] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:45.292455 32503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: No bootstrap required, opened a new log
I20260619 20:36:45.293851 32503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } }
I20260619 20:36:45.293963 32503 raft_consensus.cc:374] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:45.294009 32503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 043ffdae4c624a6797578e3edeb62d28, State: Initialized, Role: FOLLOWER
I20260619 20:36:45.294130 32503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } }
I20260619 20:36:45.294368 32509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } } }
I20260619 20:36:45.294411 32503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:45.294471 32509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 20:36:45.296032 32520 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:36:45.296087 32520 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:36:45.297721 32435 master_runner.cc:428] Detected that this master 043ffdae4c624a6797578e3edeb62d28 is joining an existing cluster
I20260619 20:36:45.297762 32435 master_runner.cc:432] Initiating AddMaster RPC to add 127.25.62.189:36729
I20260619 20:36:45.305661 32527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.298860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:36729 (local address 127.0.0.1:44058)
0619 20:36:45.299048 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.299057 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.299103 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.299227 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.299227 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.299461 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.299555 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.299557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.300412 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.300413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.301206 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.301207 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.301566 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.301567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.301623 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.302086 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.302091 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.302492 (+ 401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.304304 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.304305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.304305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.304827 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.305006 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.305007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.305008 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.305083 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.305310 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.305310 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.305346 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.305421 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.305518 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":77,"threads_started":1}
I20260619 20:36:45.305780 32506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.298781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44058 (local address 127.25.62.189:36729)
0619 20:36:45.298820 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.298822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.299116 (+ 294us) server_negotiation.cc:407] Connection header received
0619 20:36:45.299154 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.299155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.299164 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.299215 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.299631 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.300324 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.301269 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.301469 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.301551 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.302590 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.302591 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.302591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.302597 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.304220 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.304886 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.304887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.304888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.304957 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.305140 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.305141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.305141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.305243 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.305318 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.305446 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.305659 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":146,"server-negotiator.queue_time_us":21}
I20260619 20:36:45.305883 32526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.298710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.0.0.1:51406)
0619 20:36:45.298851 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.298861 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.298951 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.299124 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.299125 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.299492 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.299555 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.299557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.300192 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.300193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.301123 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.301125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.301635 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.301636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.301700 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.302248 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.302253 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.302675 (+ 422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.304391 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.304391 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.304391 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.304736 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.304916 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.304917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.304917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.305012 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.305363 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.305363 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.305405 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.305481 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.305589 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260619 20:36:45.305888 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.298747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51406 (local address 127.25.62.190:42409)
0619 20:36:45.298787 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.298790 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.298937 (+ 147us) server_negotiation.cc:407] Connection header received
0619 20:36:45.299013 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.299013 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.299025 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.299079 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.299643 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.300111 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.301204 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.301515 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.301584 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.302761 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.302762 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.302763 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.302769 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.304325 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.304799 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.304799 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.304800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.304846 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.305148 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.305149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.305149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.305278 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.305337 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.305713 (+ 376us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.305803 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901404, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.190@KRBTEST.COM for oryx/127.25.62.189@KRBTEST.COM
I20260619 20:36:45.315325 32506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.308236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44062 (local address 127.25.62.189:36729)
0619 20:36:45.308273 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.308275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.308420 (+ 145us) server_negotiation.cc:407] Connection header received
0619 20:36:45.308513 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.308514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.308521 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.308564 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.309024 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.309501 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.310568 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.310729 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.310771 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.312703 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.312705 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.312705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.312711 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.314161 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.314560 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.314561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.314562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.314625 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.314822 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.314823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.314824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.314931 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.315002 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.315173 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.315242 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:36:45.315351 32528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.308161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:36729 (local address 127.0.0.1:44062)
0619 20:36:45.308319 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.308364 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.308437 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.308582 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.308584 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.308901 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.308952 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.308954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.309574 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.309575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.310485 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.310487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.310793 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.310794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.310873 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.311255 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.311260 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.312617 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.314254 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.314261 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.314263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.314461 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.314697 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.314697 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.314698 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.314759 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.314996 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.315002 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.315050 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.315149 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.315255 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
I20260619 20:36:45.315838 31971 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.25.62.189:36729: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "043ffdae4c624a6797578e3edeb62d28"
member_type: NON_VOTER
last_known_addr {
host: "127.25.62.189"
port: 36729
}
attrs {
promote: true
}
}
dest_uuid: "d46f1eb7b6e34656a7113c47eafa65e4"
cas_config_opid_index: -1
I20260619 20:36:45.316136 31971 consensus_queue.cc:237] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } }
I20260619 20:36:45.316546 32530 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 043ffdae4c624a6797578e3edeb62d28 (127.25.62.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } } }
I20260619 20:36:45.316883 32477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:45.317651 32530 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.25.62.189:36729
I20260619 20:36:45.317734 32530 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } } }
I20260619 20:36:45.317802 32530 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:45.317799 32477 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 1 FOLLOWER]: Refusing update from remote peer d46f1eb7b6e34656a7113c47eafa65e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 20:36:45.318095 32435 master.cc:537] Master@127.25.62.189:36729 shutting down...
I20260619 20:36:45.318192 32529 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 20:36:45.318872 31958 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260619 20:36:45.319278 31958 consensus_peers.cc:487] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 -> Peer 043ffdae4c624a6797578e3edeb62d28 (127.25.62.189:36729): Couldn't send request to peer 043ffdae4c624a6797578e3edeb62d28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 20:36:45.320793 32435 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:45.320858 32435 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:45.320890 32435 tablet_replica.cc:330] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: stopping tablet replica
I20260619 20:36:45.333321 32435 master.cc:552] Master@127.25.62.189:36729 shutdown complete.
I20260619 20:36:45.333361 32435 master_runner.cc:305] Clearing existing system tablet
I20260619 20:36:45.333527 32435 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:45.333580 32435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:45.334280 32435 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:45.335007 32534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:45.335196 32435 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:45.335263 32435 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:36:45.335294 32435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/wal
uuid: "043ffdae4c624a6797578e3edeb62d28"
format_stamp: "Formatted at 2026-06-19 20:36:45 on dist-test-slave-0t40"
server_key: "9d0e8ad5049117e66569134c8948c4e8"
server_key_iv: "60dca38b6f11064e20bc6b44d0c2daf6"
server_key_version: "encryptionkey@0"
I20260619 20:36:45.335912 32435 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:45.337615 32435 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 20:36:45.337707 32435 log.cc:1196] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 20:36:45.337918 32435 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: Deleting consensus metadata
I20260619 20:36:45.338057 32435 master_runner.cc:315] Copying system tablet from 127.25.62.190:42409
I20260619 20:36:45.338618 32435 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: tablet copy: Beginning tablet copy session from remote peer at address 127.25.62.190:42409
I20260619 20:36:45.344897 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.338983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51420 (local address 127.25.62.190:42409)
0619 20:36:45.339024 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.339026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.339074 (+ 48us) server_negotiation.cc:407] Connection header received
0619 20:36:45.339148 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.339148 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.339156 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.339207 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.339559 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.340093 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.341142 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.341317 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.341370 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.342353 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.342354 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.342355 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.342361 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.343795 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.344187 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.344188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.344189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.344249 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.344453 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.344453 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.344453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.344547 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.344596 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.344760 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.344819 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:36:45.344936 32540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.338908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.0.0.1:51420)
0619 20:36:45.339021 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.339031 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.339088 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.339220 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.339221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.339409 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.339492 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.339494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.340171 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.340172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.341068 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.341071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.341386 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.341387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.341467 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.341896 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.341901 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.342274 (+ 373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.343875 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.343876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.343877 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.344093 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.344323 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.344324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.344324 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.344388 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.344605 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.344606 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.344651 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.344733 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.344831 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":64,"threads_started":1}
I20260619 20:36:45.345376 31991 tablet_copy_service.cc:140] P d46f1eb7b6e34656a7113c47eafa65e4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 043ffdae4c624a6797578e3edeb62d28 ({username='oryx', principal='oryx/127.25.62.189@KRBTEST.COM'} at 127.0.0.1:51420)
I20260619 20:36:45.345476 31991 tablet_copy_service.cc:161] P d46f1eb7b6e34656a7113c47eafa65e4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 043ffdae4c624a6797578e3edeb62d28 at {username='oryx', principal='oryx/127.25.62.189@KRBTEST.COM'} at 127.0.0.1:51420: session id = 043ffdae4c624a6797578e3edeb62d28-00000000000000000000000000000000
I20260619 20:36:45.346074 31991 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 20:36:45.346693 32435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 20:36:45.347600 32435 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: tablet copy: Starting download of 0 data blocks...
I20260619 20:36:45.347754 32435 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: tablet copy: Starting download of 1 WAL segments...
I20260619 20:36:45.348640 32435 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 20:36:45.349423 31991 tablet_copy_service.cc:342] P d46f1eb7b6e34656a7113c47eafa65e4: Request end of tablet copy session 043ffdae4c624a6797578e3edeb62d28-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.25.62.189@KRBTEST.COM'} at 127.0.0.1:51420
I20260619 20:36:45.349488 31991 tablet_copy_service.cc:433] P d46f1eb7b6e34656a7113c47eafa65e4: ending tablet copy session 043ffdae4c624a6797578e3edeb62d28-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 043ffdae4c624a6797578e3edeb62d28
I20260619 20:36:45.351763 32435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:45.352814 32541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:45.352881 32542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:45.352936 32544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:45.353108 32435 server_base.cc:1034] running on GCE node
I20260619 20:36:45.353186 32435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:45.353308 32435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:45.354452 32435 hybrid_clock.cc:648] HybridClock initialized: now 1781901405354447 us; error 26 us; skew 500 ppm
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901405, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:45.355898 32435 init.cc:377] Logged in from keytab as oryx/127.25.62.189@KRBTEST.COM (short username oryx)
I20260619 20:36:45.356168 32435 webserver.cc:460] Webserver started at http://127.25.62.189:41011/ using document root <none> and password file <none>
I20260619 20:36:45.356309 32435 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:45.356375 32435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:45.357014 32435 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:45.357585 32551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:45.357717 32435 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:45.357765 32435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/wal
uuid: "043ffdae4c624a6797578e3edeb62d28"
format_stamp: "Formatted at 2026-06-19 20:36:45 on dist-test-slave-0t40"
server_key: "9d0e8ad5049117e66569134c8948c4e8"
server_key_iv: "60dca38b6f11064e20bc6b44d0c2daf6"
server_key_version: "encryptionkey@0"
I20260619 20:36:45.357828 32435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:45.373410 32435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:45.375134 32435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:45.378436 32435 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.189:36729
I20260619 20:36:45.378660 32603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.189:36729 every 8 connection(s)
I20260619 20:36:45.378849 32435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-1/data/info.pb
I20260619 20:36:45.379733 32604 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:36:45.380262 32604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: Bootstrap starting.
I20260619 20:36:45.383077 32604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:36:45.383323 32604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: Bootstrap complete.
I20260619 20:36:45.383528 32604 raft_consensus.cc:348] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } }
I20260619 20:36:45.383616 32604 raft_consensus.cc:729] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 043ffdae4c624a6797578e3edeb62d28, State: Initialized, Role: LEARNER
I20260619 20:36:45.383656 32604 consensus_queue.cc:260] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } }
I20260619 20:36:45.383812 32604 sys_catalog.cc:564] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:45.383846 32607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } } }
I20260619 20:36:45.383955 32607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [sys.catalog]: This master's current role is: LEARNER
I20260619 20:36:45.385486 32619 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 20:36:45.391623 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.384165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44076 (local address 127.25.62.189:36729)
0619 20:36:45.384298 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.384300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.384311 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:36:45.384335 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.384336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.384353 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.384462 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.385220 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.386081 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.387070 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.387376 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.387469 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.388624 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.388626 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.388626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.388633 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.390400 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.390737 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.390739 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.390739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.390803 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.391053 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.391054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.391054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.391187 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.391293 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.391433 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.391523 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for oryx/127.25.62.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.62.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.62.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.188.keytab.
Entry for principal oryx/127.25.62.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.188.keytab.
I20260619 20:36:45.417500 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.188:45945
--webserver_interface=127.25.62.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.188.keytab
--principal=oryx/127.25.62.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:45475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:42409,127.25.62.189:36729,127.25.62.188:45945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:45.515168 32622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:45.515400 32622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:45.515435 32622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:45.516578 32622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:45.516644 32622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:45.516669 32622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:45.516690 32622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:45.516719 32622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:45.518189 32622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:45475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:42409,127.25.62.189:36729,127.25.62.188:45945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.188.keytab
--principal=oryx/127.25.62.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.188:45945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.62.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:45.518458 32622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:45.518684 32622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:45.523567 32627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:45.523726 32622 server_base.cc:1034] running on GCE node
W20260619 20:36:45.523640 32630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:45.523589 32628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:45.524080 32622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:45.524370 32622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:45.525522 32622 hybrid_clock.cc:648] HybridClock initialized: now 1781901405525495 us; error 49 us; skew 500 ppm
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901405, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:45.527910 32622 init.cc:377] Logged in from keytab as oryx/127.25.62.188@KRBTEST.COM (short username oryx)
I20260619 20:36:45.528331 32622 webserver.cc:460] Webserver started at http://127.25.62.188:38145/ using document root <none> and password file <none>
I20260619 20:36:45.528560 32622 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:45.528606 32622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:45.528769 32622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:45.529866 32622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/data/instance:
uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3"
format_stamp: "Formatted at 2026-06-19 20:36:45 on dist-test-slave-0t40"
server_key: "f2820c6407eea719bd93ab641b38c278"
server_key_iv: "ac53ca7e8d7a58a69cfedb00495b3d6f"
server_key_version: "encryptionkey@0"
I20260619 20:36:45.530230 32622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/wal/instance:
uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3"
format_stamp: "Formatted at 2026-06-19 20:36:45 on dist-test-slave-0t40"
server_key: "f2820c6407eea719bd93ab641b38c278"
server_key_iv: "ac53ca7e8d7a58a69cfedb00495b3d6f"
server_key_version: "encryptionkey@0"
I20260619 20:36:45.531814 32622 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:36:45.532604 32637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:45.532793 32622 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:45.532891 32622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/wal
uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3"
format_stamp: "Formatted at 2026-06-19 20:36:45 on dist-test-slave-0t40"
server_key: "f2820c6407eea719bd93ab641b38c278"
server_key_iv: "ac53ca7e8d7a58a69cfedb00495b3d6f"
server_key_version: "encryptionkey@0"
I20260619 20:36:45.532975 32622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:45.547793 32622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:45.549837 32622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:45.550000 32622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:45.554095 32622 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.188:45945
I20260619 20:36:45.554141 32689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.188:45945 every 8 connection(s)
I20260619 20:36:45.554499 32622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/data/info.pb
I20260619 20:36:45.555426 32690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:45.557507 32690 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:36:45.562479 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 32622
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901405, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.188@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM
I20260619 20:36:45.566608 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.558419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51430 (local address 127.25.62.190:42409)
0619 20:36:45.558479 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.558481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.558526 (+ 45us) server_negotiation.cc:407] Connection header received
0619 20:36:45.559031 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.559033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.559045 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.559116 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.559672 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.560168 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.561158 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.561334 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.561385 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.563746 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.563747 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.563748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.563754 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.565311 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.565668 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.565669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.565670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.565731 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.565945 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.565946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.565947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.566077 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.566127 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.566401 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.566497 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:36:45.566756 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.558220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.0.0.1:51430)
0619 20:36:45.558406 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.558452 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.558939 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.559176 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.559179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.559420 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.559604 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.559607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.560248 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.560248 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.561090 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.561093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.561416 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.561417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.561527 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.562018 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.562025 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.563659 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.565386 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.565391 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.565392 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.565598 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.565796 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.565796 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.565796 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.565883 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.566134 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.566137 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.566271 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.566381 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.566546 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":75,"threads_started":1}
I20260619 20:36:45.567327 32690 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.62.188@KRBTEST.COM
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901405, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.188@KRBTEST.COM for oryx/127.25.62.189@KRBTEST.COM
I20260619 20:36:45.572247 32693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.562992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36212 (local address 127.25.62.188:45945)
0619 20:36:45.563147 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.563151 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.563174 (+ 23us) server_negotiation.cc:407] Connection header received
0619 20:36:45.563196 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.563197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.563230 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.563330 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.564067 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.565028 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.566006 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.566384 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.566445 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.569148 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.569155 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.569155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.569162 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.570819 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.571143 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.571146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.571147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.571203 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.571417 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.571418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.571418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.571601 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.571739 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.572058 (+ 319us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.572138 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260619 20:36:45.574131 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.567573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:36729 (local address 127.0.0.1:44080)
0619 20:36:45.567673 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.567683 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.567750 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.567921 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.567922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.568061 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.568109 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.568110 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.568742 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.568744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.569387 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.569390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.569793 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.569794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.569863 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.570299 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.570303 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.571536 (+ 1233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.573380 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.573380 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.573381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.573532 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.573705 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.573705 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.573705 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.573743 (+ 38us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.573928 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.573929 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.573970 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.574019 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.574077 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260619 20:36:45.574153 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.567629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44080 (local address 127.25.62.189:36729)
0619 20:36:45.567683 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.567686 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.567721 (+ 35us) server_negotiation.cc:407] Connection header received
0619 20:36:45.567805 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.567806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.567821 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.567913 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.568199 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.568660 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.569477 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.569705 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.569792 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.571613 (+ 1821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.571614 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.571614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.571620 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.573301 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.573608 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.573609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.573609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.573666 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.573775 (+ 109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.573776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.573776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.573861 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.573921 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.574034 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.574094 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 20:36:45.574595 32690 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901405, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.188@KRBTEST.COM for oryx/127.25.62.188@KRBTEST.COM
I20260619 20:36:45.580605 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.574874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:45945 (local address 127.0.0.1:36218)
0619 20:36:45.574921 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.574931 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.574967 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.575106 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.575106 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.575230 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.575252 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.575253 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.575844 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.575845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.576297 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.576298 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.576652 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.576652 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.576698 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.577099 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.577104 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.578138 (+ 1034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.579651 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.579652 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.579652 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.579807 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.580007 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.580007 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.580008 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.580060 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.580292 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.580293 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.580327 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.580394 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.580488 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 20:36:45.580653 32693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.574941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36218 (local address 127.25.62.188:45945)
0619 20:36:45.574971 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.574973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.574981 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:45.575017 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.575018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.575027 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.575101 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.575329 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.575764 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.576385 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.576573 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.576653 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.578226 (+ 1573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.578227 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.578228 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.578234 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.579554 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.579886 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.579887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.579888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.579938 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.580129 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.580130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.580130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.580229 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.580299 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.580420 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.580535 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":18}
I20260619 20:36:45.582055 32690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: Bootstrap starting.
I20260619 20:36:45.582896 32690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:45.583180 32690 log.cc:824] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:45.583937 32690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: No bootstrap required, opened a new log
I20260619 20:36:45.585772 32690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } }
I20260619 20:36:45.585937 32690 raft_consensus.cc:374] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:45.585987 32690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 694e1bf8ae0c4f94a8e26b6fff3c31f3, State: Initialized, Role: FOLLOWER
I20260619 20:36:45.586120 32690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } }
I20260619 20:36:45.586428 32690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:45.586382 32695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } } }
I20260619 20:36:45.586508 32695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 20:36:45.588058 32706 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:36:45.588132 32706 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:36:45.590451 32622 master_runner.cc:428] Detected that this master 694e1bf8ae0c4f94a8e26b6fff3c31f3 is joining an existing cluster
I20260619 20:36:45.590517 32622 master_runner.cc:432] Initiating AddMaster RPC to add 127.25.62.188:45945
I20260619 20:36:45.598376 32693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.591746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36234 (local address 127.25.62.188:45945)
0619 20:36:45.591788 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.591789 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.591980 (+ 191us) server_negotiation.cc:407] Connection header received
0619 20:36:45.592075 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.592075 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.592085 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.592118 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.592622 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.593325 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.594430 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.594677 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.594722 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.595862 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.595863 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.595863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.595869 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.597280 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.597659 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.597659 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.597660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.597706 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.597926 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.597927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.597927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.598031 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.598076 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.598260 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.598320 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:36:45.598501 32713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.591763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:45945 (local address 127.0.0.1:36234)
0619 20:36:45.591920 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.591930 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.592003 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.592175 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.592176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.592456 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.592529 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.592532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.593438 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.593439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.594326 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.594329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.594801 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.594802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.594873 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.595281 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.595287 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.595771 (+ 484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.597357 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.597358 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.597358 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.597582 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.597770 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.597771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.597772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.597845 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.598094 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.598095 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.598154 (+ 59us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.598245 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.598392 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":123,"thread_start_us":78,"threads_started":1}
I20260619 20:36:45.613524 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.591529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51434 (local address 127.25.62.190:42409)
0619 20:36:45.591574 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.591576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.591803 (+ 227us) server_negotiation.cc:407] Connection header received
0619 20:36:45.591958 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.591959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.591971 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.592004 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.592615 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.593397 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.594465 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.594770 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.594811 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.610463 (+ 15652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.610465 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.610465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.610473 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.612006 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.612372 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.612373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.612373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.612466 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.612682 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.612683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.612684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.612855 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.612942 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.613323 (+ 381us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.613414 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:36:45.613708 32714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.591486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:36729 (local address 127.0.0.1:44090)
0619 20:36:45.591897 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.591907 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.591975 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.592145 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.592146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.592416 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.592523 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.592524 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.593483 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.593484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.594384 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.594386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.594887 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.594888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.594960 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.609703 (+ 14743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.609707 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.610322 (+ 615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.612447 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.612448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.612448 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.612639 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.612933 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.612934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.612934 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.612994 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.613344 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.613344 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.613388 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.613480 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.613595 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":30,"spinlock_wait_cycles":9600,"thread_start_us":107,"threads_started":1}
I20260619 20:36:45.613744 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.591610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44090 (local address 127.25.62.189:36729)
0619 20:36:45.591647 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.591648 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.591952 (+ 304us) server_negotiation.cc:407] Connection header received
0619 20:36:45.592044 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.592046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.592055 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.592085 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.592614 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.593385 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.594479 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.594776 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.594812 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.610441 (+ 15629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.610443 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.610444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.610453 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.612360 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.612777 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.612778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.612778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.612841 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.613093 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.613093 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.613094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.613240 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.613311 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.613524 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.613634 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:36:45.614313 32712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.591485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.0.0.1:51434)
0619 20:36:45.591710 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.591721 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.591862 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.592077 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.592078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.592414 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.592523 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.592524 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.593483 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.593484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.594366 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.594368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.594889 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.594890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.594983 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.609785 (+ 14802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.609789 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.610322 (+ 533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.612114 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.612115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.612116 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.612295 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.612533 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.612534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.612534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.612609 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.613109 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.613109 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.613155 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.614053 (+ 898us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.614166 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":43,"thread_start_us":73,"threads_started":1}
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901404, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.190@KRBTEST.COM for oryx/127.25.62.188@KRBTEST.COM
I20260619 20:36:45.623283 32693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.616407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36242 (local address 127.25.62.188:45945)
0619 20:36:45.616461 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.616463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.616471 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:45.616501 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.616502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.616518 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.616567 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.616917 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.617467 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.618450 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.618616 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.618662 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.620659 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.620661 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.620661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.620668 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.622203 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.622522 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.622523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.622523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.622584 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.622760 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.622761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.622761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.622894 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.622969 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.623122 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.623197 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 20:36:45.623485 32528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.616286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:45945 (local address 127.0.0.1:36242)
0619 20:36:45.616333 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.616346 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.616406 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.616594 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.616595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.616765 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.616826 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.616827 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.617543 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.617544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.618361 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.618364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.618686 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.618687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.618756 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.619139 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.619144 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.620579 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.622283 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.622284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.622284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.622448 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.622654 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.622655 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.622655 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.622702 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.622974 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.622975 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.623019 (+ 44us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.623263 (+ 244us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.623403 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260619 20:36:45.623474 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 32033
I20260619 20:36:45.624346 31971 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.25.62.188:45945: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3"
member_type: NON_VOTER
last_known_addr {
host: "127.25.62.188"
port: 45945
}
attrs {
promote: true
}
}
dest_uuid: "d46f1eb7b6e34656a7113c47eafa65e4"
cas_config_opid_index: 5
I20260619 20:36:45.624540 31971 consensus_queue.cc:237] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true } }
I20260619 20:36:45.624872 32530 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 694e1bf8ae0c4f94a8e26b6fff3c31f3 (127.25.62.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true } } }
I20260619 20:36:45.625893 32530 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.25.62.188:45945
I20260619 20:36:45.625967 32530 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true } } }
I20260619 20:36:45.626030 32530 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:45.626895 32664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:45.627800 32664 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 1 FOLLOWER]: Refusing update from remote peer d46f1eb7b6e34656a7113c47eafa65e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 20:36:45.628103 32529 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:45.629081 32622 master.cc:537] Master@127.25.62.188:45945 shutting down...
I20260619 20:36:45.629803 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:33507
--local_ip_for_outbound_sockets=127.25.62.129
--tserver_master_addrs=127.25.62.190:42409
--webserver_port=34757
--webserver_interface=127.25.62.129
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.129.keytab
--principal=oryx/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:45475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.62.190:42409,127.25.62.189:36729
--tserver_master_addrs=127.25.62.190:42409,127.25.62.189:36729,127.25.62.188:45945 with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:36:45.630435 32664 raft_consensus.cc:858] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true } }
W20260619 20:36:45.630515 32664 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true } }
I20260619 20:36:45.631016 32664 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 043ffdae4c624a6797578e3edeb62d28 (127.25.62.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } } }
I20260619 20:36:45.631744 32528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.624952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:36729 (local address 127.0.0.1:44102)
0619 20:36:45.624983 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.624995 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.625057 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.625246 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.625247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.625394 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.625437 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.625439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.626583 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.626584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.627363 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.627366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.627867 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.627868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.627921 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.628329 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.628333 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.628684 (+ 351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.630622 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.630622 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.630623 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.630916 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.631120 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.631121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.631121 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.631173 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.631466 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.631467 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.631507 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.631566 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.631652 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260619 20:36:45.632347 32695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d46f1eb7b6e34656a7113c47eafa65e4. Latest consensus state: current_term: 1 leader_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } } }
I20260619 20:36:45.632443 32695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260619 20:36:45.632479 32695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } } }
I20260619 20:36:45.632524 32695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260619 20:36:45.632550 32695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } } }
I20260619 20:36:45.632582 32695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260619 20:36:45.632674 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.625080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44102 (local address 127.25.62.189:36729)
0619 20:36:45.625125 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.625127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.625134 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:36:45.625153 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.625154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.625165 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.625230 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.625518 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.626249 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.627468 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.627763 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.627820 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.628780 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.628781 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.628781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.628788 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.630530 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.630996 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.630997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.630997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.631053 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.631236 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.631237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.631237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.631372 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.631439 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.631889 (+ 450us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.631963 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 20:36:45.632858 32578 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 1 LEARNER]: Refusing update from remote peer d46f1eb7b6e34656a7113c47eafa65e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 20:36:45.633073 32529 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 20:36:45.633706 32578 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 694e1bf8ae0c4f94a8e26b6fff3c31f3 (127.25.62.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true } } }
I20260619 20:36:45.634440 32715 mvcc.cc:205] Tried to move back new op lower bound from 7298668153334784000 to 7298668153258565632. Current Snapshot: MvccSnapshot[applied={T|T < 7298668153268809728}]
I20260619 20:36:45.635001 32607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d46f1eb7b6e34656a7113c47eafa65e4. Latest consensus state: current_term: 1 leader_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true } } }
I20260619 20:36:45.635106 32607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [sys.catalog]: This master's current role is: LEARNER
I20260619 20:36:45.635146 32529 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4: attempting to promote NON_VOTER 043ffdae4c624a6797578e3edeb62d28 to VOTER
I20260619 20:36:45.635152 32607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: NON_VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: true } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true } } }
I20260619 20:36:45.635203 32607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [sys.catalog]: This master's current role is: LEARNER
I20260619 20:36:45.635321 32529 consensus_queue.cc:237] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true } }
I20260619 20:36:45.635726 32578 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 1 LEARNER]: Refusing update from remote peer d46f1eb7b6e34656a7113c47eafa65e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 20:36:45.635880 32529 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260619 20:36:45.636453 32529 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 043ffdae4c624a6797578e3edeb62d28 (127.25.62.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true } } }
W20260619 20:36:45.636952 31955 consensus_peers.cc:487] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 -> Peer 694e1bf8ae0c4f94a8e26b6fff3c31f3 (127.25.62.188:45945): Couldn't send request to peer 694e1bf8ae0c4f94a8e26b6fff3c31f3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 20:36:45.637146 32622 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260619 20:36:45.637247 32622 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260619 20:36:45.637284 32622 tablet_replica.cc:330] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: stopping tablet replica
I20260619 20:36:45.637328 32529 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true } } }
I20260619 20:36:45.637398 32529 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:45.637457 32529 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true } } }
I20260619 20:36:45.637502 32529 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:45.637892 32578 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 043ffdae4c624a6797578e3edeb62d28 (127.25.62.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true } } }
I20260619 20:36:45.638850 32724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true } } }
I20260619 20:36:45.638933 32724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:36:45.651535 32622 master.cc:552] Master@127.25.62.188:45945 shutdown complete.
I20260619 20:36:45.651592 32622 master_runner.cc:305] Clearing existing system tablet
I20260619 20:36:45.651733 32622 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:45.651784 32622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:45.652555 32622 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:36:45.653170 32727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:45.653327 32622 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:45.653378 32622 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:36:45.653402 32622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/wal
uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3"
format_stamp: "Formatted at 2026-06-19 20:36:45 on dist-test-slave-0t40"
server_key: "f2820c6407eea719bd93ab641b38c278"
server_key_iv: "ac53ca7e8d7a58a69cfedb00495b3d6f"
server_key_version: "encryptionkey@0"
I20260619 20:36:45.654135 32622 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:45.655527 32622 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 20:36:45.655596 32622 log.cc:1196] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 20:36:45.655818 32622 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: Deleting consensus metadata
I20260619 20:36:45.656039 32622 master_runner.cc:315] Copying system tablet from 127.25.62.190:42409
I20260619 20:36:45.656641 32622 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: tablet copy: Beginning tablet copy session from remote peer at address 127.25.62.190:42409
I20260619 20:36:45.664466 32733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.656945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.0.0.1:51448)
0619 20:36:45.657161 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.657173 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.657238 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.657382 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.657383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.657585 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.657643 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.657645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.658641 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.658642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.659492 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.659494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.660038 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.660039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.660103 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.661086 (+ 983us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.661092 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.661506 (+ 414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.663324 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.663325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.663326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.663561 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.663795 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.663796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.663796 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.663862 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.664141 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.664142 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.664189 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.664260 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.664360 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":63,"threads_started":1}
I20260619 20:36:45.664469 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.657015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51448 (local address 127.25.62.190:42409)
0619 20:36:45.657121 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.657123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.657269 (+ 146us) server_negotiation.cc:407] Connection header received
0619 20:36:45.657297 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.657297 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.657309 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.657378 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.657726 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.658549 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.659608 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.659907 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.659952 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.661605 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.661606 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.661606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.661614 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.663219 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.663658 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.663659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.663659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.663721 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.663929 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.663930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.663930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.664071 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.664145 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.664298 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.664372 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 20:36:45.665135 31991 tablet_copy_service.cc:140] P d46f1eb7b6e34656a7113c47eafa65e4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 694e1bf8ae0c4f94a8e26b6fff3c31f3 ({username='oryx', principal='oryx/127.25.62.188@KRBTEST.COM'} at 127.0.0.1:51448)
I20260619 20:36:45.665196 31991 tablet_copy_service.cc:161] P d46f1eb7b6e34656a7113c47eafa65e4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 694e1bf8ae0c4f94a8e26b6fff3c31f3 at {username='oryx', principal='oryx/127.25.62.188@KRBTEST.COM'} at 127.0.0.1:51448: session id = 694e1bf8ae0c4f94a8e26b6fff3c31f3-00000000000000000000000000000000
I20260619 20:36:45.665753 31991 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 20:36:45.666241 32622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 20:36:45.667419 32622 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: tablet copy: Starting download of 0 data blocks...
I20260619 20:36:45.667716 32622 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: tablet copy: Starting download of 1 WAL segments...
I20260619 20:36:45.668548 32622 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 20:36:45.669469 31991 tablet_copy_service.cc:342] P d46f1eb7b6e34656a7113c47eafa65e4: Request end of tablet copy session 694e1bf8ae0c4f94a8e26b6fff3c31f3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.25.62.188@KRBTEST.COM'} at 127.0.0.1:51448
I20260619 20:36:45.669526 31991 tablet_copy_service.cc:433] P d46f1eb7b6e34656a7113c47eafa65e4: ending tablet copy session 694e1bf8ae0c4f94a8e26b6fff3c31f3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 694e1bf8ae0c4f94a8e26b6fff3c31f3
I20260619 20:36:45.672324 32622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:45.673563 32734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:45.673542 32737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:45.677219 32622 server_base.cc:1034] running on GCE node
W20260619 20:36:45.677910 32735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:45.678079 32622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:45.678225 32622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:45.679368 32622 hybrid_clock.cc:648] HybridClock initialized: now 1781901405679353 us; error 29 us; skew 500 ppm
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901405, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:45.680967 32622 init.cc:377] Logged in from keytab as oryx/127.25.62.188@KRBTEST.COM (short username oryx)
I20260619 20:36:45.681288 32622 webserver.cc:460] Webserver started at http://127.25.62.188:45645/ using document root <none> and password file <none>
I20260619 20:36:45.681427 32622 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:45.681468 32622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:45.682205 32622 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:36:45.682796 32744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:45.682914 32622 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:36:45.682956 32622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/wal
uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3"
format_stamp: "Formatted at 2026-06-19 20:36:45 on dist-test-slave-0t40"
server_key: "f2820c6407eea719bd93ab641b38c278"
server_key_iv: "ac53ca7e8d7a58a69cfedb00495b3d6f"
server_key_version: "encryptionkey@0"
I20260619 20:36:45.683015 32622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:45.696663 32622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:45.698457 32622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:45.701534 32622 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.188:45945
I20260619 20:36:45.701884 32622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/master-2/data/info.pb
I20260619 20:36:45.704512 328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.188:45945 every 8 connection(s)
I20260619 20:36:45.705307 329 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:36:45.705884 329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: Bootstrap starting.
I20260619 20:36:45.708133 329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:36:45.708338 329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: Bootstrap complete.
I20260619 20:36:45.708506 329 raft_consensus.cc:348] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true } }
I20260619 20:36:45.708596 329 raft_consensus.cc:729] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 694e1bf8ae0c4f94a8e26b6fff3c31f3, State: Initialized, Role: LEARNER
I20260619 20:36:45.708630 329 consensus_queue.cc:260] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true } }
I20260619 20:36:45.708806 329 sys_catalog.cc:564] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:45.708796 332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true } } }
I20260619 20:36:45.709069 332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: This master's current role is: LEARNER
I20260619 20:36:45.710353 343 catalog_manager.cc:1259] Loaded cluster ID: e87fcace73ff40c79a43b072c099bda4
I20260619 20:36:45.710453 343 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: loading cluster ID for follower catalog manager: success
I20260619 20:36:45.711899 343 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: acquiring CA information for follower catalog manager: success
I20260619 20:36:45.712247 343 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 20:36:45.732314 32717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:45.732558 32717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:45.732600 32717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:45.733855 32717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:45.733932 32717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:45.733980 32717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:36:45.735512 32717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:45475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.129.keytab
--principal=oryx/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:33507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=34757
--tserver_master_addrs=127.25.62.190:42409,127.25.62.189:36729,127.25.62.188:45945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:45.735780 32717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:45.736020 32717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:45.741137 348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:45.741123 349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:45.741351 351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:45.741271 32717 server_base.cc:1034] running on GCE node
I20260619 20:36:45.741655 32717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:45.741958 32717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:45.743121 32717 hybrid_clock.cc:648] HybridClock initialized: now 1781901405743108 us; error 30 us; skew 500 ppm
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901405, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:45.745707 32717 init.cc:377] Logged in from keytab as oryx/127.25.62.129@KRBTEST.COM (short username oryx)
I20260619 20:36:45.746188 32717 webserver.cc:460] Webserver started at http://127.25.62.129:34757/ using document root <none> and password file <none>
I20260619 20:36:45.746431 32717 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:45.746493 32717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:45.747915 32717 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:45.748558 358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:45.748773 32717 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:36:45.748883 32717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "afca6bfafe0342f6a6e7167ae9632164"
format_stamp: "Formatted at 2026-06-19 20:36:44 on dist-test-slave-0t40"
server_key: "4c37c76e82bee9c058e1ab96557ba4f4"
server_key_iv: "e88559d44ecd6e4d8dbd1c20294cc7a3"
server_key_version: "encryptionkey@0"
I20260619 20:36:45.749312 32717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:45.760979 32717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:45.763428 32717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:45.763598 32717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:45.763832 32717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:45.764173 32717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:45.764228 32717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:45.764274 32717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:45.764317 32717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:45.770876 32717 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:33507
I20260619 20:36:45.770906 471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:33507 every 8 connection(s)
I20260619 20:36:45.771310 32717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901405, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.189@KRBTEST.COM
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901405, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901405, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.188@KRBTEST.COM
I20260619 20:36:45.781425 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 32717
I20260619 20:36:45.781598 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 32167
I20260619 20:36:45.782878 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.772424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:55923 (local address 127.25.62.189:36729)
0619 20:36:45.772485 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.772487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.772495 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:45.773049 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.773050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.773064 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.773097 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.774183 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.774685 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.775932 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.776221 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.776296 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.778905 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.778906 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.778906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.778914 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.780532 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.781118 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.781120 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.781120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.781183 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.781501 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.781503 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.781503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.781661 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.781712 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.782668 (+ 956us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.782763 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 20:36:45.783171 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.772344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:54955 (local address 127.25.62.190:42409)
0619 20:36:45.772425 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.772427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.772438 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:36:45.773019 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.773020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.773084 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.773151 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.774236 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.774750 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.775805 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.776108 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.776178 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.779975 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.779977 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.779977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.779986 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.781814 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.782181 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.782182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.782182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.782240 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.782478 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.782479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.782480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.782626 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.782700 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.782990 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.783077 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 20:36:45.783515 476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.772059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.129:54955)
0619 20:36:45.772233 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.772275 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.772905 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.773242 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.773247 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.773790 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.774066 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.774070 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.774869 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.774870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.775686 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.775688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.776250 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.776251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.776413 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.777108 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.777114 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.779650 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.781897 (+ 2247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.781898 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.781899 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.782100 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.782319 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.782320 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.782320 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.782393 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.782788 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.782789 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.782828 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.783330 (+ 502us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.783425 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":40,"thread_start_us":95,"threads_started":1}
I20260619 20:36:45.783514 477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.772060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:45945 (local address 127.25.62.129:56955)
0619 20:36:45.772333 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.772345 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.772925 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.773262 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.773263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.773842 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.774066 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.774071 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.775971 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.775972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.776645 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.776647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.777120 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.777121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.777198 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.777629 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.777634 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.779992 (+ 2358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.782122 (+ 2130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.782123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.782124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.782345 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.782551 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.782552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.782552 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.782609 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.782882 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.782883 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.782925 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.783004 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.783221 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":12,"thread_start_us":87,"threads_started":1}
I20260619 20:36:45.783625 479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.772507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:56955 (local address 127.25.62.188:45945)
0619 20:36:45.772605 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.772607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.772614 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:36:45.773115 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.773116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.773129 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.773221 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.774856 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.775867 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.776723 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.776989 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.777110 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.780101 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.780102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.780103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.780111 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.782015 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.782425 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.782426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.782426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.782482 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.782675 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.782675 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.782676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.782811 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.783348 (+ 537us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.783392 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.783482 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":56,"threads_started":1}
I20260619 20:36:45.783898 478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.772060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:36729 (local address 127.25.62.129:55923)
0619 20:36:45.772344 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.772352 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.772925 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.773529 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.773529 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.773869 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.774066 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.774071 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.774776 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.774777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.775846 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.775849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.776303 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.776304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.776411 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.777016 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.777025 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.778803 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.780645 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.780652 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.780654 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.780991 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.781243 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.781244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.781245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.781398 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.782319 (+ 921us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.782327 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.782498 (+ 171us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.783008 (+ 510us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.783311 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":347,"thread_start_us":55,"threads_started":1}
I20260619 20:36:45.784080 473 heartbeater.cc:347] Connected to a master server at 127.25.62.190:42409
I20260619 20:36:45.784169 473 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:45.784355 473 heartbeater.cc:511] Master 127.25.62.190:42409 requested a full tablet report, sending...
I20260619 20:36:45.784782 31971 ts_manager.cc:194] Re-registered known tserver with Master: afca6bfafe0342f6a6e7167ae9632164 (127.25.62.129:33507)
I20260619 20:36:45.785352 31971 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:54955
I20260619 20:36:45.787500 472 heartbeater.cc:347] Connected to a master server at 127.25.62.188:45945
I20260619 20:36:45.787549 472 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:45.787627 472 heartbeater.cc:511] Master 127.25.62.188:45945 requested a full tablet report, sending...
I20260619 20:36:45.787752 474 heartbeater.cc:347] Connected to a master server at 127.25.62.189:36729
I20260619 20:36:45.787786 474 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:45.787828 474 heartbeater.cc:511] Master 127.25.62.189:36729 requested a full tablet report, sending...
I20260619 20:36:45.788069 32761 ts_manager.cc:194] Registered new tserver with Master: afca6bfafe0342f6a6e7167ae9632164 (127.25.62.129:33507)
I20260619 20:36:45.788172 32568 ts_manager.cc:194] Registered new tserver with Master: afca6bfafe0342f6a6e7167ae9632164 (127.25.62.129:33507)
I20260619 20:36:45.789598 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:36761
--local_ip_for_outbound_sockets=127.25.62.130
--tserver_master_addrs=127.25.62.190:42409
--webserver_port=36401
--webserver_interface=127.25.62.130
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.130.keytab
--principal=oryx/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:45475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.62.190:42409,127.25.62.189:36729
--tserver_master_addrs=127.25.62.190:42409,127.25.62.189:36729,127.25.62.188:45945 with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:45.868821 480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:45.869024 480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:45.869093 480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:45.870260 480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:45.870298 480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:45.870347 480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:36:45.871775 480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:45475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.130.keytab
--principal=oryx/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:36761
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=36401
--tserver_master_addrs=127.25.62.190:42409,127.25.62.189:36729,127.25.62.188:45945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:45.872008 480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:45.872242 480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:45.877346 485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:45.877379 488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:45.877372 486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:45.877447 480 server_base.cc:1034] running on GCE node
I20260619 20:36:45.877701 480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:45.877961 480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:45.879153 480 hybrid_clock.cc:648] HybridClock initialized: now 1781901405879133 us; error 30 us; skew 500 ppm
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901405, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:45.881690 480 init.cc:377] Logged in from keytab as oryx/127.25.62.130@KRBTEST.COM (short username oryx)
I20260619 20:36:45.882120 480 webserver.cc:460] Webserver started at http://127.25.62.130:36401/ using document root <none> and password file <none>
I20260619 20:36:45.882335 480 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:45.882433 480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:45.883785 480 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:45.884521 495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:45.884687 480 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:45.884760 480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "a576e1317b724daeaea1183f19e0548e"
format_stamp: "Formatted at 2026-06-19 20:36:44 on dist-test-slave-0t40"
server_key: "216045aa00d9ca23c3e3ba7036278806"
server_key_iv: "e7aa066eb8a391b1e11ae571f93c13ea"
server_key_version: "encryptionkey@0"
I20260619 20:36:45.885231 480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:45.900364 480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:45.902670 480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:45.902812 480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:45.903074 480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:45.903378 480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:45.903409 480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:45.903432 480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:45.903479 480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:45.909911 480 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:36761
I20260619 20:36:45.909971 608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:36761 every 8 connection(s)
I20260619 20:36:45.910389 480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:36:45.914072 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 480
I20260619 20:36:45.914230 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 32301
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901405, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.189@KRBTEST.COM
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901405, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.188@KRBTEST.COM
Jun 19 20:36:45 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901405, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM
I20260619 20:36:45.921473 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.911369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:54783 (local address 127.25.62.189:36729)
0619 20:36:45.911430 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.911432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.912174 (+ 742us) server_negotiation.cc:407] Connection header received
0619 20:36:45.912256 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.912257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.912267 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.912307 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.913173 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.913651 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.914795 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.914994 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.915079 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.917411 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.917413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.917413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.917421 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.919626 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.920266 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.920268 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.920268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.920326 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.920611 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.920612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.920612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.920750 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.920792 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.921248 (+ 456us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.921335 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 20:36:45.922312 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.911106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:36729 (local address 127.25.62.130:54783)
0619 20:36:45.912117 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.912127 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.912198 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.912319 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.912319 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.912740 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.913081 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.913087 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.913721 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.913721 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.914725 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.914728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.915087 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.915088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.915214 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.915668 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.915677 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.917313 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.919702 (+ 2389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.919709 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.919711 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.920159 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.920403 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.920404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.920404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.920538 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.920881 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.920887 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.921092 (+ 205us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.921773 (+ 681us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.922049 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":985,"mutex_wait_us":284,"thread_start_us":65,"threads_started":1}
I20260619 20:36:45.922318 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:43649
--local_ip_for_outbound_sockets=127.25.62.131
--tserver_master_addrs=127.25.62.190:42409
--webserver_port=40419
--webserver_interface=127.25.62.131
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.131.keytab
--principal=oryx/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:45475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.25.62.190:42409,127.25.62.189:36729
--tserver_master_addrs=127.25.62.190:42409,127.25.62.189:36729,127.25.62.188:45945 with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:36:45.922603 479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.911099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:51151 (local address 127.25.62.188:45945)
0619 20:36:45.911146 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.911149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.911554 (+ 405us) server_negotiation.cc:407] Connection header received
0619 20:36:45.912057 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.912059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.912071 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.912137 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.914514 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.915311 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.916313 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.916667 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.916737 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.919382 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.919392 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.919393 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.919400 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.921122 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.921503 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.921504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.921505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.921618 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.921838 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.921839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.921840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.922076 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.922149 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.922402 (+ 253us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.922495 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:36:45.922946 614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.911105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:45945 (local address 127.25.62.130:51151)
0619 20:36:45.911288 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.911328 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.911944 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.912720 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.912721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.912943 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.913081 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.913087 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.915432 (+ 2345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.915433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.916221 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.916224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.916749 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.916750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.916817 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.917881 (+ 1064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.917886 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.919284 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.921197 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.921197 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.921198 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.921417 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.921699 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.921700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.921700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.921763 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.922208 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.922209 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.922257 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.922717 (+ 460us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.922820 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":2,"thread_start_us":88,"threads_started":1}
I20260619 20:36:45.924232 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.911375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:57439 (local address 127.25.62.190:42409)
0619 20:36:45.911479 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:45.911481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:45.911490 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:36:45.912057 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:45.912058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:45.912071 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:45.912132 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:45.913730 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.914416 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.917859 (+ 3443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.918221 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.918292 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.921164 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:45.921165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:45.921166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:45.921173 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:45.923001 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.923317 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.923318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.923319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.923380 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:45.923596 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:45.923597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:45.923598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:45.923734 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:45.923787 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:45.924037 (+ 250us) server_negotiation.cc:299] Negotiation successful
0619 20:36:45.924119 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 20:36:45.924566 613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:45.911106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.130:57439)
0619 20:36:45.911389 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:45.911398 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:45.911942 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:45.912217 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:45.912222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:45.912937 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:45.913082 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.913089 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.917103 (+ 4014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.917104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.917756 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:45.917758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:45.918310 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:45.918311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:45.918380 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:45.919001 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:45.919006 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:45.921059 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:45.923088 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.923089 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.923089 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.923237 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.923447 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:45.923448 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:45.923448 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:45.923511 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:45.923819 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:45.923820 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:45.923862 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 20:36:45.924348 (+ 486us) client_negotiation.cc:240] Negotiation successful
0619 20:36:45.924457 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":2,"thread_start_us":88,"threads_started":1}
I20260619 20:36:45.924649 611 heartbeater.cc:347] Connected to a master server at 127.25.62.189:36729
I20260619 20:36:45.924746 611 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:45.924805 609 heartbeater.cc:347] Connected to a master server at 127.25.62.188:45945
I20260619 20:36:45.924834 609 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:45.924934 611 heartbeater.cc:511] Master 127.25.62.189:36729 requested a full tablet report, sending...
I20260619 20:36:45.924933 609 heartbeater.cc:511] Master 127.25.62.188:45945 requested a full tablet report, sending...
I20260619 20:36:45.925154 610 heartbeater.cc:347] Connected to a master server at 127.25.62.190:42409
I20260619 20:36:45.925195 610 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:45.925256 610 heartbeater.cc:511] Master 127.25.62.190:42409 requested a full tablet report, sending...
I20260619 20:36:45.925444 32761 ts_manager.cc:194] Registered new tserver with Master: a576e1317b724daeaea1183f19e0548e (127.25.62.130:36761)
I20260619 20:36:45.925463 31971 ts_manager.cc:194] Re-registered known tserver with Master: a576e1317b724daeaea1183f19e0548e (127.25.62.130:36761)
I20260619 20:36:45.925505 32568 ts_manager.cc:194] Registered new tserver with Master: a576e1317b724daeaea1183f19e0548e (127.25.62.130:36761)
I20260619 20:36:45.925905 31971 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:57439
W20260619 20:36:46.004232 616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:46.004436 616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:46.004462 616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:46.005806 616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:46.005847 616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:46.005875 616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:36:46.007280 616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:45475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.131.keytab
--principal=oryx/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:43649
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=40419
--tserver_master_addrs=127.25.62.190:42409,127.25.62.189:36729,127.25.62.188:45945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:46.007472 616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:46.007647 616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:46.012792 621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:46.012854 624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:46.012861 622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:46.013015 616 server_base.cc:1034] running on GCE node
I20260619 20:36:46.013247 616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:46.013492 616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:46.014642 616 hybrid_clock.cc:648] HybridClock initialized: now 1781901406014632 us; error 30 us; skew 500 ppm
Jun 19 20:36:46 dist-test-slave-0t40 krb5kdc[31926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901406, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:46.017127 616 init.cc:377] Logged in from keytab as oryx/127.25.62.131@KRBTEST.COM (short username oryx)
I20260619 20:36:46.017594 616 webserver.cc:460] Webserver started at http://127.25.62.131:40419/ using document root <none> and password file <none>
I20260619 20:36:46.017833 616 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:46.017910 616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:46.019200 616 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:46.019812 631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:46.019996 616 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:46.020089 616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "7589a1cd5a8d467cbd7d9610a2db3936"
format_stamp: "Formatted at 2026-06-19 20:36:45 on dist-test-slave-0t40"
server_key: "6b7a16daaac7050ebdf820ab21b2ca3d"
server_key_iv: "5151a54270942315fd139b3d29603fb4"
server_key_version: "encryptionkey@0"
I20260619 20:36:46.020421 616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:46.034233 616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:46.036198 616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:46.036357 616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:46.036578 616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:46.036878 616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:46.036931 616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:46.036965 616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:46.036993 616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:46.042974 616 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:43649
I20260619 20:36:46.043011 744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:43649 every 8 connection(s)
I20260619 20:36:46.043363 616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:36:46.048413 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 616
Jun 19 20:36:46 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901406, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM
Jun 19 20:36:46 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901406, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.188@KRBTEST.COM
Jun 19 20:36:46 dist-test-slave-0t40 krb5kdc[31926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901406, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.189@KRBTEST.COM
I20260619 20:36:46.056252 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.044218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:39879 (local address 127.25.62.189:36729)
0619 20:36:46.044315 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:46.044318 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:46.044531 (+ 213us) server_negotiation.cc:407] Connection header received
0619 20:36:46.045845 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:46.045846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:46.045857 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:46.045910 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:46.046494 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.046972 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.048267 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.048551 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.048591 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.052629 (+ 4038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:46.052630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:46.052631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:46.052638 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:46.054696 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:46.055143 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:46.055144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:46.055144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:46.055202 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:46.055496 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:46.055497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:46.055497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:46.055623 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:46.055681 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:46.056086 (+ 405us) server_negotiation.cc:299] Negotiation successful
0619 20:36:46.056165 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 20:36:46.056388 479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.044279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:40485 (local address 127.25.62.188:45945)
0619 20:36:46.044330 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:46.044333 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:46.044677 (+ 344us) server_negotiation.cc:407] Connection header received
0619 20:36:46.045644 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:46.045645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:46.045658 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:46.045706 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:46.046514 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.047261 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.048310 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.048616 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.048654 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.053218 (+ 4564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:46.053219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:46.053220 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:46.053226 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:46.055101 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:46.055516 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:46.055517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:46.055518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:46.055577 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:46.055764 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:46.055765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:46.055766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:46.055890 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:46.055951 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:46.056194 (+ 243us) server_negotiation.cc:299] Negotiation successful
0619 20:36:46.056278 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:36:46.056406 749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.044073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.131:50841)
0619 20:36:46.044392 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:46.044437 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:46.045530 (+ 1093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:46.045800 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:46.045802 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:46.046270 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:46.046404 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:46.046410 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.047360 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.047361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:46.048075 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:46.048077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.048645 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.048647 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:46.048794 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.049535 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:46.049545 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:46.051417 (+ 1872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:46.054392 (+ 2975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:46.054399 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:46.054401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:46.055058 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:46.055273 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:46.055274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:46.055274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:46.055418 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:46.055710 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:46.055716 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:46.055870 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 20:36:46.056006 (+ 136us) client_negotiation.cc:240] Negotiation successful
0619 20:36:46.056177 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":61,"thread_start_us":98,"threads_started":1}
I20260619 20:36:46.056689 751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.044073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:45945 (local address 127.25.62.131:40485)
0619 20:36:46.044631 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:46.044639 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:46.045539 (+ 900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:46.045799 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:46.045802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:46.046251 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:46.046405 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:46.046410 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.047356 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.047357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:46.048223 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:46.048225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.048721 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.048721 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:46.048795 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.049753 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:46.049759 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:46.053128 (+ 3369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:46.055194 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:46.055196 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:46.055196 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:46.055420 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:46.055641 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:46.055642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:46.055642 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:46.055698 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:46.055980 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:46.055981 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:46.056026 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 20:36:46.056097 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 20:36:46.056574 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":431,"thread_start_us":82,"threads_started":1}
I20260619 20:36:46.056730 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.044367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:50841 (local address 127.25.62.190:42409)
0619 20:36:46.044412 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:46.044413 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:46.044530 (+ 117us) server_negotiation.cc:407] Connection header received
0619 20:36:46.045715 (+ 1185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:46.045716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:46.045726 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:46.045789 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:46.046514 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.047276 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.048183 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.048510 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.048576 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.051517 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:46.051518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:46.051519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:46.051526 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:46.053724 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:46.055143 (+ 1419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:46.055144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:46.055144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:46.055202 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:46.055496 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:46.055497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:46.055498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:46.055621 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:46.055682 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:46.056549 (+ 867us) server_negotiation.cc:299] Negotiation successful
0619 20:36:46.056653 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:36:46.057111 745 heartbeater.cc:347] Connected to a master server at 127.25.62.188:45945
I20260619 20:36:46.057121 746 heartbeater.cc:347] Connected to a master server at 127.25.62.190:42409
I20260619 20:36:46.057226 746 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:46.057478 746 heartbeater.cc:511] Master 127.25.62.190:42409 requested a full tablet report, sending...
I20260619 20:36:46.057809 750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.044073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:36729 (local address 127.25.62.131:39879)
0619 20:36:46.044436 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:46.044445 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:46.045530 (+ 1085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:46.046038 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:46.046039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:46.046215 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:46.046404 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:46.046408 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.047062 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.047063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:46.048143 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:46.048145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.048669 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.048670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:46.048795 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.049862 (+ 1067us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:46.049866 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:46.052540 (+ 2674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:46.054781 (+ 2241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:46.054781 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:46.054782 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:46.055058 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:46.055273 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:46.055274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:46.055274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:46.055418 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:46.055710 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:46.055716 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:46.055870 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 20:36:46.056006 (+ 136us) client_negotiation.cc:240] Negotiation successful
0619 20:36:46.057613 (+ 1607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":1478,"thread_start_us":89,"threads_started":1}
I20260619 20:36:46.057888 31971 ts_manager.cc:194] Re-registered known tserver with Master: 7589a1cd5a8d467cbd7d9610a2db3936 (127.25.62.131:43649)
I20260619 20:36:46.058362 31971 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:50841
I20260619 20:36:46.057225 745 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:46.058722 745 heartbeater.cc:511] Master 127.25.62.188:45945 requested a full tablet report, sending...
I20260619 20:36:46.058763 747 heartbeater.cc:347] Connected to a master server at 127.25.62.189:36729
I20260619 20:36:46.058809 747 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:46.058874 747 heartbeater.cc:511] Master 127.25.62.189:36729 requested a full tablet report, sending...
I20260619 20:36:46.059049 32761 ts_manager.cc:194] Registered new tserver with Master: 7589a1cd5a8d467cbd7d9610a2db3936 (127.25.62.131:43649)
I20260619 20:36:46.059171 32568 ts_manager.cc:194] Registered new tserver with Master: 7589a1cd5a8d467cbd7d9610a2db3936 (127.25.62.131:43649)
I20260619 20:36:46.061585 759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.049780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36258 (local address 127.25.62.188:45945)
0619 20:36:46.049872 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:46.049874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:46.050177 (+ 303us) server_negotiation.cc:407] Connection header received
0619 20:36:46.050201 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:46.050201 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:46.050213 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:46.050272 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:46.051093 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.051864 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.053151 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.053483 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.053558 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.058873 (+ 5315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:46.058874 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:46.058875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:46.058881 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:46.060444 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:46.060764 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:46.060766 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:46.060766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:46.060818 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:46.061075 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:46.061076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:46.061076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:46.061190 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:46.061294 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:46.061399 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 20:36:46.061481 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":50,"threads_started":1}
I20260619 20:36:46.062458 758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.049723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51450 (local address 127.25.62.190:42409)
0619 20:36:46.051144 (+ 1421us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:46.051146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:46.051153 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:36:46.051172 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:46.051173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:46.051184 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:46.052006 (+ 822us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:46.052047 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.052847 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.054010 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.054311 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.054629 (+ 318us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.055807 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:46.055808 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:46.055808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:46.055814 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:46.061126 (+ 5312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:46.061648 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:46.061649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:46.061650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:46.061709 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:46.061930 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:46.061931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:46.061931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:46.062080 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:46.062179 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:46.062264 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 20:36:46.062344 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1390,"thread_start_us":75,"threads_started":1}
I20260619 20:36:46.064865 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.054293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44106 (local address 127.25.62.189:36729)
0619 20:36:46.056322 (+ 2029us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:46.056323 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:46.057124 (+ 801us) server_negotiation.cc:407] Connection header received
0619 20:36:46.057219 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:46.057220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:46.057230 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:46.057259 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:46.057911 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.058576 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.059899 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.060213 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.060288 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.062327 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:46.062328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:46.062328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:46.062334 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:46.063859 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:46.064197 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:46.064198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:46.064199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:46.064257 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:46.064427 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:46.064428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:46.064428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:46.064546 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:46.064634 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:46.064715 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 20:36:46.064781 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2015,"thread_start_us":54,"threads_started":1}
I20260619 20:36:46.064210 31971 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51450:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:36:46.066179 31971 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:36:46.073382 766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.069502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:43649 (local address 127.0.0.1:49316)
0619 20:36:46.069641 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:46.069650 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:46.069697 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:46.070039 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:46.070040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:46.070043 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:46.070092 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:46.070093 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.071379 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.071380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:46.072314 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:46.072316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.073195 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.073195 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:46.073241 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.073256 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 20:36:46.073301 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 20:36:46.073331 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":101,"threads_started":1}
I20260619 20:36:46.073426 768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.069699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49316 (local address 127.25.62.131:43649)
0619 20:36:46.069839 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:46.069843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:46.069852 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:36:46.069889 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:46.069891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:46.069908 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:46.070164 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:46.070211 (+ 47us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.071280 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.072395 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.073118 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.073190 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.073260 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:46.073315 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 20:36:46.073349 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
I20260619 20:36:46.074096 679 tablet_service.cc:1467] Processing CreateTablet for tablet 8a3c058934f74a49941b69e696e80a6f (DEFAULT_TABLE table=test-table [id=e46ef24dd4914247aed3c01b8a98731a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:46.074450 679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a3c058934f74a49941b69e696e80a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:46.074661 32528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.069472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:36761 (local address 127.0.0.1:48188)
0619 20:36:46.069507 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:46.069518 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:46.069558 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:46.070090 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:46.070091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:46.070093 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:46.070134 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:46.070136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.071343 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.071343 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:46.072465 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:46.072468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.074412 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.074413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:46.074467 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.074469 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:36:46.074548 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 20:36:46.074586 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":2}
I20260619 20:36:46.074908 767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.069538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:33507 (local address 127.0.0.1:37778)
0619 20:36:46.069950 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:46.069961 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:46.070004 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:46.070385 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:46.070386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:46.070388 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:46.070422 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:46.070423 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.071233 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.071234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:46.072467 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:46.072468 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.074672 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.074673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:46.074733 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.074736 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:36:46.074789 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:36:46.074836 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":63,"thread_start_us":90,"threads_started":1}
I20260619 20:36:46.075131 769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.069745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37778 (local address 127.25.62.129:33507)
0619 20:36:46.070233 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:46.070238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:46.070247 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:36:46.070289 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:46.070291 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:46.070303 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:46.070390 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:46.070492 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.071150 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.072721 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.073459 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.074546 (+ 1087us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.074641 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:46.075012 (+ 371us) server_negotiation.cc:299] Negotiation successful
0619 20:36:46.075056 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":85,"threads_started":1}
I20260619 20:36:46.075146 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.069775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48188 (local address 127.25.62.130:36761)
0619 20:36:46.069921 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:46.069926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:46.069937 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:36:46.069973 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:46.069975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:46.069990 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:46.070092 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:46.070192 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.071259 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.072551 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.074332 (+ 1781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.074408 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.074476 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:46.074619 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 20:36:46.074661 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":77,"threads_started":1}
I20260619 20:36:46.075747 406 tablet_service.cc:1467] Processing CreateTablet for tablet 8a3c058934f74a49941b69e696e80a6f (DEFAULT_TABLE table=test-table [id=e46ef24dd4914247aed3c01b8a98731a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:46.075763 543 tablet_service.cc:1467] Processing CreateTablet for tablet 8a3c058934f74a49941b69e696e80a6f (DEFAULT_TABLE table=test-table [id=e46ef24dd4914247aed3c01b8a98731a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:46.076052 543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a3c058934f74a49941b69e696e80a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:46.076052 406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a3c058934f74a49941b69e696e80a6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:46.077525 771 tablet_bootstrap.cc:492] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936: Bootstrap starting.
I20260619 20:36:46.078181 771 tablet_bootstrap.cc:654] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:46.078480 771 log.cc:824] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:46.079144 771 tablet_bootstrap.cc:492] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936: No bootstrap required, opened a new log
I20260619 20:36:46.079222 771 ts_tablet_manager.cc:1397] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:36:46.079584 774 tablet_bootstrap.cc:492] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164: Bootstrap starting.
I20260619 20:36:46.080076 774 tablet_bootstrap.cc:654] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:46.080360 774 log.cc:824] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:46.080684 773 tablet_bootstrap.cc:492] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e: Bootstrap starting.
I20260619 20:36:46.081207 774 tablet_bootstrap.cc:492] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164: No bootstrap required, opened a new log
I20260619 20:36:46.081311 773 tablet_bootstrap.cc:654] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:46.081311 774 ts_tablet_manager.cc:1397] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:36:46.081257 771 raft_consensus.cc:348] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } } peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } }
I20260619 20:36:46.081451 771 raft_consensus.cc:374] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:46.081492 771 raft_consensus.cc:729] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7589a1cd5a8d467cbd7d9610a2db3936, State: Initialized, Role: FOLLOWER
I20260619 20:36:46.081619 771 consensus_queue.cc:260] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } } peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } }
I20260619 20:36:46.081666 773 log.cc:824] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:46.081921 771 ts_tablet_manager.cc:1428] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:36:46.082114 746 heartbeater.cc:503] Master 127.25.62.190:42409 was elected leader, sending a full tablet report...
I20260619 20:36:46.082310 773 tablet_bootstrap.cc:492] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e: No bootstrap required, opened a new log
I20260619 20:36:46.082373 773 ts_tablet_manager.cc:1397] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 20:36:46.084769 773 raft_consensus.cc:348] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } } peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } }
I20260619 20:36:46.084769 774 raft_consensus.cc:348] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } } peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } }
I20260619 20:36:46.084923 773 raft_consensus.cc:374] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:46.084923 774 raft_consensus.cc:374] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:46.084959 773 raft_consensus.cc:729] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a576e1317b724daeaea1183f19e0548e, State: Initialized, Role: FOLLOWER
I20260619 20:36:46.084969 774 raft_consensus.cc:729] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afca6bfafe0342f6a6e7167ae9632164, State: Initialized, Role: FOLLOWER
I20260619 20:36:46.085090 773 consensus_queue.cc:260] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } } peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } }
I20260619 20:36:46.085115 774 consensus_queue.cc:260] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } } peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } }
I20260619 20:36:46.085343 773 ts_tablet_manager.cc:1428] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 20:36:46.085500 610 heartbeater.cc:503] Master 127.25.62.190:42409 was elected leader, sending a full tablet report...
I20260619 20:36:46.085726 774 ts_tablet_manager.cc:1428] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 20:36:46.086158 473 heartbeater.cc:503] Master 127.25.62.190:42409 was elected leader, sending a full tablet report...
I20260619 20:36:46.156651 767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.154145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:45945 (local address 127.0.0.1:36260)
0619 20:36:46.154210 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:46.154224 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:46.154263 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:46.154487 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:46.154488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:46.154490 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:46.154538 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:46.154539 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.155174 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.155174 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:46.155876 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:46.155878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.156444 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.156445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:46.156498 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.156502 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:46.156552 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 20:36:46.156589 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 20:36:46.156656 759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.154262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36260 (local address 127.25.62.188:45945)
0619 20:36:46.154326 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:46.154328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:46.154336 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:46.154363 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:46.154364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:46.154386 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:46.154455 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:46.154606 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.155106 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.155970 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.156363 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.156422 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.156475 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:46.156565 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 20:36:46.156603 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 20:36:46.157006 32763 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260619 20:36:46.157466 332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d46f1eb7b6e34656a7113c47eafa65e4. Latest consensus state: current_term: 1 leader_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true } } }
I20260619 20:36:46.157586 332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: This master's current role is: LEARNER
W20260619 20:36:46.161804 612 tablet.cc:2369] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:36:46.173553 778 raft_consensus.cc:482] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:36:46.173668 778 raft_consensus.cc:504] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } } peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } }
I20260619 20:36:46.174037 778 leader_election.cc:288] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7589a1cd5a8d467cbd7d9610a2db3936 (127.25.62.131:43649), afca6bfafe0342f6a6e7167ae9632164 (127.25.62.129:33507)
I20260619 20:36:46.177912 768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.174298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:35997 (local address 127.25.62.131:43649)
0619 20:36:46.174357 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:46.174359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:46.174368 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:36:46.174400 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:46.174401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:46.174411 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:46.174486 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:46.174721 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.175467 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.176925 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.177508 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.177588 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.177596 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:46.177809 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 20:36:46.177846 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 20:36:46.177912 769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.174346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:52187 (local address 127.25.62.129:33507)
0619 20:36:46.174391 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:46.174392 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:46.174417 (+ 25us) server_negotiation.cc:407] Connection header received
0619 20:36:46.174440 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:46.174441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:46.174449 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:46.174500 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:46.174721 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.175463 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.176925 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.177514 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.177588 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.177596 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:46.177809 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 20:36:46.177846 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:36:46.177948 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.174243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:33507 (local address 127.25.62.130:52187)
0619 20:36:46.174327 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:46.174336 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:46.174368 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:46.174516 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:46.174517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:46.174518 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:46.174636 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:46.174638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.175551 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.175552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:46.176826 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:46.176827 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.177618 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.177620 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:46.177722 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.177726 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:46.177792 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 20:36:46.177843 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":177}
I20260619 20:36:46.177968 613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.174221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:43649 (local address 127.25.62.130:35997)
0619 20:36:46.174278 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:46.174290 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:46.174340 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:46.174516 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:46.174517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:46.174518 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:46.174636 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:46.174638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.175551 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.175552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:46.176819 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:46.176822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.177618 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.177620 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:46.177721 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.177726 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:36:46.177792 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 20:36:46.177843 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":39}
I20260619 20:36:46.178303 426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a3c058934f74a49941b69e696e80a6f" candidate_uuid: "a576e1317b724daeaea1183f19e0548e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afca6bfafe0342f6a6e7167ae9632164" is_pre_election: true
I20260619 20:36:46.178462 426 raft_consensus.cc:2454] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a576e1317b724daeaea1183f19e0548e in term 0.
I20260619 20:36:46.178459 699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a3c058934f74a49941b69e696e80a6f" candidate_uuid: "a576e1317b724daeaea1183f19e0548e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" is_pre_election: true
I20260619 20:36:46.178629 699 raft_consensus.cc:2454] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a576e1317b724daeaea1183f19e0548e in term 0.
I20260619 20:36:46.178668 499 leader_election.cc:302] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a576e1317b724daeaea1183f19e0548e, afca6bfafe0342f6a6e7167ae9632164; no voters:
I20260619 20:36:46.178768 778 raft_consensus.cc:2781] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:36:46.178818 778 raft_consensus.cc:482] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:36:46.178843 778 raft_consensus.cc:3037] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:46.179528 778 raft_consensus.cc:504] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } } peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } }
I20260619 20:36:46.179667 778 leader_election.cc:288] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [CANDIDATE]: Term 1 election: Requested vote from peers 7589a1cd5a8d467cbd7d9610a2db3936 (127.25.62.131:43649), afca6bfafe0342f6a6e7167ae9632164 (127.25.62.129:33507)
I20260619 20:36:46.179806 699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a3c058934f74a49941b69e696e80a6f" candidate_uuid: "a576e1317b724daeaea1183f19e0548e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7589a1cd5a8d467cbd7d9610a2db3936"
I20260619 20:36:46.179813 426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a3c058934f74a49941b69e696e80a6f" candidate_uuid: "a576e1317b724daeaea1183f19e0548e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afca6bfafe0342f6a6e7167ae9632164"
I20260619 20:36:46.179908 699 raft_consensus.cc:3037] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:46.179905 426 raft_consensus.cc:3037] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:46.180626 426 raft_consensus.cc:2454] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a576e1317b724daeaea1183f19e0548e in term 1.
I20260619 20:36:46.180682 699 raft_consensus.cc:2454] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a576e1317b724daeaea1183f19e0548e in term 1.
I20260619 20:36:46.180778 499 leader_election.cc:302] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a576e1317b724daeaea1183f19e0548e, afca6bfafe0342f6a6e7167ae9632164; no voters:
I20260619 20:36:46.180878 778 raft_consensus.cc:2781] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:46.181075 778 raft_consensus.cc:686] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [term 1 LEADER]: Becoming Leader. State: Replica: a576e1317b724daeaea1183f19e0548e, State: Running, Role: LEADER
I20260619 20:36:46.181182 778 consensus_queue.cc:237] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } } peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } }
I20260619 20:36:46.182058 31971 catalog_manager.cc:5495] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e reported cstate change: term changed from 0 to 1, leader changed from <none> to a576e1317b724daeaea1183f19e0548e (127.25.62.130). New cstate: current_term: 1 leader_uuid: "a576e1317b724daeaea1183f19e0548e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } health_report { overall_health: UNKNOWN } } }
I20260619 20:36:46.182662 32725 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: NON_VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: true } } }
I20260619 20:36:46.182750 32725 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:46.183275 32530 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4: attempting to promote NON_VOTER 694e1bf8ae0c4f94a8e26b6fff3c31f3 to VOTER
I20260619 20:36:46.183401 32530 consensus_queue.cc:237] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: false } }
I20260619 20:36:46.183868 32578 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 1 FOLLOWER]: Refusing update from remote peer d46f1eb7b6e34656a7113c47eafa65e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260619 20:36:46.183951 32763 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 1 LEARNER]: Refusing update from remote peer d46f1eb7b6e34656a7113c47eafa65e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260619 20:36:46.184163 32725 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260619 20:36:46.184237 785 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260619 20:36:46.184819 32725 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 694e1bf8ae0c4f94a8e26b6fff3c31f3 (127.25.62.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: false } } }
I20260619 20:36:46.185001 32763 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 694e1bf8ae0c4f94a8e26b6fff3c31f3 (127.25.62.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: false } } }
I20260619 20:36:46.185250 32578 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 694e1bf8ae0c4f94a8e26b6fff3c31f3 (127.25.62.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: false } } }
I20260619 20:36:46.185899 32725 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: false } } }
I20260619 20:36:46.185966 32725 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:46.186017 32725 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: false } } }
I20260619 20:36:46.186059 32725 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:46.186128 332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: false } } }
I20260619 20:36:46.186209 332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:36:46.186133 785 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: false } } }
I20260619 20:36:46.186244 785 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:46.186520 787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: false } } }
I20260619 20:36:46.186616 787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:36:46.190399 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.188140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48194 (local address 127.25.62.130:36761)
0619 20:36:46.188195 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:46.188197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:46.188208 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:36:46.188261 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:46.188262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:46.188285 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:46.188340 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:46.188447 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.188934 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.189730 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.189917 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.189973 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.190130 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:46.190215 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:46.190273 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:46.190304 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 20:36:46.190320 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 20:36:46.192029 699 raft_consensus.cc:1261] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936 [term 1 FOLLOWER]: Refusing update from remote peer a576e1317b724daeaea1183f19e0548e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:46.192054 426 raft_consensus.cc:1261] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164 [term 1 FOLLOWER]: Refusing update from remote peer a576e1317b724daeaea1183f19e0548e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:46.192384 783 consensus_queue.cc:1035] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [LEADER]: Connected to new peer: Peer: permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:46.192509 778 consensus_queue.cc:1035] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:46.195861 790 mvcc.cc:205] Tried to move back new op lower bound from 7298668159759941632 to 7298668159718715392. Current Snapshot: MvccSnapshot[applied={T|T < 7298668159759941632}]
I20260619 20:36:46.196787 791 mvcc.cc:205] Tried to move back new op lower bound from 7298668159759941632 to 7298668159718715392. Current Snapshot: MvccSnapshot[applied={T|T < 7298668159759941632 or (T in {7298668159759941632})}]
I20260619 20:36:46.198779 31971 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:46.198851 31971 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:46.200379 31971 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d46f1eb7b6e34656a7113c47eafa65e4: Sending DeleteTablet for 3 replicas of tablet 8a3c058934f74a49941b69e696e80a6f
I20260619 20:36:46.200663 406 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a3c058934f74a49941b69e696e80a6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:46 UTC) from {username='oryx', principal='oryx/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:37778
I20260619 20:36:46.200910 798 tablet_replica.cc:330] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164: stopping tablet replica
I20260619 20:36:46.201016 798 raft_consensus.cc:2229] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:46.201079 679 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a3c058934f74a49941b69e696e80a6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:46 UTC) from {username='oryx', principal='oryx/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:49316
I20260619 20:36:46.201144 798 raft_consensus.cc:2258] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:46.201124 543 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a3c058934f74a49941b69e696e80a6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:46 UTC) from {username='oryx', principal='oryx/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:48188
I20260619 20:36:46.201272 799 tablet_replica.cc:330] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936: stopping tablet replica
I20260619 20:36:46.201354 799 raft_consensus.cc:2229] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:46.201356 800 tablet_replica.cc:330] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e: stopping tablet replica
I20260619 20:36:46.201442 799 raft_consensus.cc:2258] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:46.201447 800 raft_consensus.cc:2229] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:36:46.201560 800 raft_consensus.cc:2258] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:46.201895 799 ts_tablet_manager.cc:1905] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:46.202072 798 ts_tablet_manager.cc:1905] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:46.202590 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 31939
I20260619 20:36:46.203583 799 ts_tablet_manager.cc:1918] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:36:46.203653 799 log.cc:1196] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/8a3c058934f74a49941b69e696e80a6f
I20260619 20:36:46.203933 799 ts_tablet_manager.cc:1939] T 8a3c058934f74a49941b69e696e80a6f P 7589a1cd5a8d467cbd7d9610a2db3936: Deleting consensus metadata
I20260619 20:36:46.204483 798 ts_tablet_manager.cc:1918] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:36:46.204559 798 log.cc:1196] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-0/wal/wals/8a3c058934f74a49941b69e696e80a6f
I20260619 20:36:46.204808 798 ts_tablet_manager.cc:1939] T 8a3c058934f74a49941b69e696e80a6f P afca6bfafe0342f6a6e7167ae9632164: Deleting consensus metadata
I20260619 20:36:46.205102 800 ts_tablet_manager.cc:1905] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:46.207010 800 ts_tablet_manager.cc:1918] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:36:46.207086 800 log.cc:1196] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/8a3c058934f74a49941b69e696e80a6f
I20260619 20:36:46.207337 800 ts_tablet_manager.cc:1939] T 8a3c058934f74a49941b69e696e80a6f P a576e1317b724daeaea1183f19e0548e: Deleting consensus metadata
W20260619 20:36:46.209340 359 connection.cc:535] server connection from 127.0.0.1:37778 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 20:36:46.209618 497 connection.cc:535] server connection from 127.0.0.1:48188 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 20:36:46.209856 634 connection.cc:535] server connection from 127.0.0.1:49316 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 20:36:46.218384 479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.210796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36282 (local address 127.25.62.188:45945)
0619 20:36:46.210840 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:46.210843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:46.210966 (+ 123us) server_negotiation.cc:407] Connection header received
0619 20:36:46.211068 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:46.211069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:46.211084 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:46.211149 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:46.211837 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.212576 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.213528 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.213752 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.213812 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.215427 (+ 1615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:46.215428 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:46.215428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:46.215436 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:46.217089 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:46.217459 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:46.217460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:46.217460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:46.217521 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:46.217731 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:46.217732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:46.217732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:46.217866 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:46.217961 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:46.218191 (+ 230us) server_negotiation.cc:299] Negotiation successful
0619 20:36:46.218266 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:36:46.218552 32608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.210984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44120 (local address 127.25.62.189:36729)
0619 20:36:46.211028 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:46.211030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:46.211143 (+ 113us) server_negotiation.cc:407] Connection header received
0619 20:36:46.211260 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:46.211262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:46.211273 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:46.211329 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:46.211896 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.212356 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.213381 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.213691 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.213766 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.215707 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:46.215709 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:46.215709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:46.215716 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:46.217354 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:46.217749 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:46.217750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:46.217750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:46.217801 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:46.217997 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:46.217998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:46.217998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:46.218119 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:46.218183 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:46.218371 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 20:36:46.218444 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:36:46.221426 759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:46.202017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36276 (local address 127.25.62.188:45945)
0619 20:36:46.202218 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:46.202220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:46.203550 (+ 1330us) server_negotiation.cc:407] Connection header received
0619 20:36:46.203656 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:46.203657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:46.203669 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:46.203700 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:46.204501 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.205259 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.217146 (+ 11887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:46.217471 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:46.217510 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:46.218923 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:46.218924 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:46.218924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:46.218931 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:46.220418 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:46.220719 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:46.220721 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:46.220721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:46.220779 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:46.220954 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:46.220954 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:46.220955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:46.221098 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:46.221191 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:46.221260 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 20:36:46.221334 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
I20260619 20:36:46.385999 32619 catalog_manager.cc:1259] Loaded cluster ID: e87fcace73ff40c79a43b072c099bda4
I20260619 20:36:46.386058 32619 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: loading cluster ID for follower catalog manager: success
I20260619 20:36:46.386830 32619 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: acquiring CA information for follower catalog manager: success
I20260619 20:36:46.387130 32619 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:36:47.197288 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.196974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.130:59877)
0619 20:36:47.197150 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.197229 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":49,"threads_started":1}
I20260619 20:36:47.197314 810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.197007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.129:48917)
0619 20:36:47.197189 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.197261 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":107,"threads_started":1}
I20260619 20:36:47.197364 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.196959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.131:48193)
0619 20:36:47.197268 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.197324 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":61,"threads_started":1}
W20260619 20:36:47.197496 746 heartbeater.cc:644] Failed to heartbeat to 127.25.62.190:42409 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
W20260619 20:36:47.197496 610 heartbeater.cc:644] Failed to heartbeat to 127.25.62.190:42409 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
W20260619 20:36:47.197516 473 heartbeater.cc:644] Failed to heartbeat to 127.25.62.190:42409 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
I20260619 20:36:47.700536 811 raft_consensus.cc:482] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d46f1eb7b6e34656a7113c47eafa65e4)
I20260619 20:36:47.700654 811 raft_consensus.cc:504] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: false } }
I20260619 20:36:47.700904 811 leader_election.cc:288] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d46f1eb7b6e34656a7113c47eafa65e4 (127.25.62.190:42409), 043ffdae4c624a6797578e3edeb62d28 (127.25.62.189:36729)
I20260619 20:36:47.701243 812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.700988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.0.0.1:51924)
0619 20:36:47.701134 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.701190 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
W20260619 20:36:47.701380 32746 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
I20260619 20:36:47.701417 815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d46f1eb7b6e34656a7113c47eafa65e4)
I20260619 20:36:47.701676 815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: false } }
I20260619 20:36:47.701794 812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.701716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.0.0.1:51936)
0619 20:36:47.701749 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.701771 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260619 20:36:47.701848 32746 leader_election.cc:334] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d46f1eb7b6e34656a7113c47eafa65e4 (127.25.62.190:42409): Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
I20260619 20:36:47.702406 818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.702221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.0.0.1:51942)
0619 20:36:47.702317 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.702362 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":52,"threads_started":1}
W20260619 20:36:47.702502 32553 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
I20260619 20:36:47.702597 815 leader_election.cc:288] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d46f1eb7b6e34656a7113c47eafa65e4 (127.25.62.190:42409), 694e1bf8ae0c4f94a8e26b6fff3c31f3 (127.25.62.188:45945)
I20260619 20:36:47.702947 818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.702842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.0.0.1:51952)
0619 20:36:47.702895 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.702922 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 20:36:47.703084 32553 leader_election.cc:334] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d46f1eb7b6e34656a7113c47eafa65e4 (127.25.62.190:42409): Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
I20260619 20:36:47.704607 813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.701085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:36729 (local address 127.0.0.1:44134)
0619 20:36:47.701186 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.701197 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:47.701254 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:47.701496 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:47.701497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:47.701500 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:47.701563 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:47.701564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.702458 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.702459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:47.703729 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:47.703731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.704423 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.704424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:47.704480 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:47.704485 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:36:47.704530 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 20:36:47.704562 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":3,"thread_start_us":54,"threads_started":1}
I20260619 20:36:47.704628 814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.701131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44134 (local address 127.25.62.189:36729)
0619 20:36:47.701270 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:47.701271 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:47.701279 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:47.701325 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:47.701326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:47.701349 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:47.701441 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:47.701658 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.702357 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.703804 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.704330 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.704412 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:47.704475 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:47.704536 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 20:36:47.704570 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":57,"threads_started":1}
I20260619 20:36:47.704819 32578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "043ffdae4c624a6797578e3edeb62d28" is_pre_election: true
I20260619 20:36:47.704926 32578 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 694e1bf8ae0c4f94a8e26b6fff3c31f3 in term 1.
I20260619 20:36:47.705125 32748 leader_election.cc:302] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 043ffdae4c624a6797578e3edeb62d28, 694e1bf8ae0c4f94a8e26b6fff3c31f3; no voters: d46f1eb7b6e34656a7113c47eafa65e4
I20260619 20:36:47.705267 811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 20:36:47.705324 811 raft_consensus.cc:482] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d46f1eb7b6e34656a7113c47eafa65e4)
I20260619 20:36:47.705356 811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 1 FOLLOWER]: Advancing to term 2
I20260619 20:36:47.705794 819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.702379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36286 (local address 127.25.62.188:45945)
0619 20:36:47.702641 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:47.702643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:47.702649 (+ 6us) server_negotiation.cc:407] Connection header received
0619 20:36:47.702669 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:47.702670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:47.702680 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:47.702792 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:47.702965 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.703641 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.704957 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.705503 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.705559 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:47.705567 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:47.705723 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 20:36:47.705755 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":90,"threads_started":1}
I20260619 20:36:47.705806 817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.702229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:45945 (local address 127.0.0.1:36286)
0619 20:36:47.702485 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.702496 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:47.702548 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:47.702826 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:47.702827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:47.702829 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:47.702867 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:47.702868 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.703717 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.703717 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:47.704855 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:47.704857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.705583 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.705584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:47.705644 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:47.705648 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:47.705709 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 20:36:47.705749 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":39,"threads_started":1}
I20260619 20:36:47.706116 32763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "043ffdae4c624a6797578e3edeb62d28" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" is_pre_election: true
I20260619 20:36:47.706288 811 raft_consensus.cc:504] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: false } }
I20260619 20:36:47.706415 32763 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 043ffdae4c624a6797578e3edeb62d28 in current term 2: Already voted for candidate 694e1bf8ae0c4f94a8e26b6fff3c31f3 in this term.
I20260619 20:36:47.706427 811 leader_election.cc:288] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [CANDIDATE]: Term 2 election: Requested vote from peers d46f1eb7b6e34656a7113c47eafa65e4 (127.25.62.190:42409), 043ffdae4c624a6797578e3edeb62d28 (127.25.62.189:36729)
I20260619 20:36:47.706558 32578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "043ffdae4c624a6797578e3edeb62d28"
I20260619 20:36:47.706615 813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.706500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.0.0.1:51962)
0619 20:36:47.706569 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.706590 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 20:36:47.706635 32578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 1 FOLLOWER]: Advancing to term 2
I20260619 20:36:47.706908 813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.706829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.0.0.1:51964)
0619 20:36:47.706874 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.706894 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 20:36:47.706983 32746 leader_election.cc:334] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d46f1eb7b6e34656a7113c47eafa65e4 (127.25.62.190:42409): Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
I20260619 20:36:47.707384 32578 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 694e1bf8ae0c4f94a8e26b6fff3c31f3 in term 2.
I20260619 20:36:47.707553 32552 leader_election.cc:302] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 043ffdae4c624a6797578e3edeb62d28; no voters: 694e1bf8ae0c4f94a8e26b6fff3c31f3, d46f1eb7b6e34656a7113c47eafa65e4
I20260619 20:36:47.707569 32748 leader_election.cc:302] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 043ffdae4c624a6797578e3edeb62d28, 694e1bf8ae0c4f94a8e26b6fff3c31f3; no voters: d46f1eb7b6e34656a7113c47eafa65e4
I20260619 20:36:47.707675 815 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:36:47.707687 811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 20:36:47.707809 811 raft_consensus.cc:686] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [term 2 LEADER]: Becoming Leader. State: Replica: 694e1bf8ae0c4f94a8e26b6fff3c31f3, State: Running, Role: LEADER
I20260619 20:36:47.707876 811 consensus_queue.cc:237] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: false } }
I20260619 20:36:47.708243 821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 694e1bf8ae0c4f94a8e26b6fff3c31f3. Latest consensus state: current_term: 2 leader_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: false } } }
I20260619 20:36:47.708334 821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:47.708597 823 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:47.709002 823 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:47.709111 823 catalog_manager.cc:1259] Loaded cluster ID: e87fcace73ff40c79a43b072c099bda4
I20260619 20:36:47.709144 823 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:47.709311 823 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:47.709386 823 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: Loaded TSK: 0
I20260619 20:36:47.709549 823 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:36:47.752028 32760 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:36:47.753749 32760 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:36:47.754899 813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.754770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.0.0.1:51978)
0619 20:36:47.754827 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.754863 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 20:36:47.754951 32578 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [term 2 FOLLOWER]: Refusing update from remote peer 694e1bf8ae0c4f94a8e26b6fff3c31f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260619 20:36:47.755257 813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.755132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.0.0.1:51986)
0619 20:36:47.755202 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.755235 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 20:36:47.755205 821 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260619 20:36:47.755347 32746 consensus_peers.cc:487] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 -> Peer d46f1eb7b6e34656a7113c47eafa65e4 (127.25.62.190:42409): Couldn't send request to peer d46f1eb7b6e34656a7113c47eafa65e4. Status: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 20:36:47.755890 815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 694e1bf8ae0c4f94a8e26b6fff3c31f3. Latest consensus state: current_term: 2 leader_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: false } } }
I20260619 20:36:47.755991 815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:36:47.756414 821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: false } } }
I20260619 20:36:47.756505 821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:47.756501 815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "d46f1eb7b6e34656a7113c47eafa65e4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42409 } } peers { permanent_uuid: "043ffdae4c624a6797578e3edeb62d28" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 36729 } attrs { promote: false } } peers { permanent_uuid: "694e1bf8ae0c4f94a8e26b6fff3c31f3" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 45945 } attrs { promote: false } } }
I20260619 20:36:47.756595 815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043ffdae4c624a6797578e3edeb62d28 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:36:47.756726 825 mvcc.cc:205] Tried to move back new op lower bound from 7298668166161567744 to 7298668165972066304. Current Snapshot: MvccSnapshot[applied={T|T < 7298668166161567744}]
I20260619 20:36:47.761826 830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.758351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49324 (local address 127.25.62.131:43649)
0619 20:36:47.758469 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:47.758471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:47.758478 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:36:47.758505 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:47.758505 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:47.758517 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:47.758734 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:47.758779 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.759607 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.760894 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.761481 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.761548 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:47.761558 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:47.761706 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 20:36:47.761750 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
I20260619 20:36:47.761831 812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.758328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:33507 (local address 127.0.0.1:37786)
0619 20:36:47.758454 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.758466 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:47.758513 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:47.758933 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:47.758934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:47.758943 (+ 9us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:47.758986 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:47.758987 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.759834 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.759834 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:47.760900 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:47.760901 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.761527 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.761528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:47.761593 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:47.761597 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:47.761678 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 20:36:47.761722 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260619 20:36:47.761840 813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.758292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:43649 (local address 127.0.0.1:49324)
0619 20:36:47.758334 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.758344 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:47.758396 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:47.758606 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:47.758607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:47.758609 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:47.758663 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:47.758665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.759682 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.759683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:47.760797 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:47.760799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.761589 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.761590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:47.761648 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:47.761652 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:47.761698 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 20:36:47.761741 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 20:36:47.762065 832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.758483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37786 (local address 127.25.62.129:33507)
0619 20:36:47.758657 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:47.758659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:47.758667 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:47.758693 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:47.758694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:47.758774 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:47.758893 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:47.759062 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.759761 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.760978 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.761420 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.761526 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:47.761539 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:47.761915 (+ 376us) server_negotiation.cc:299] Negotiation successful
0619 20:36:47.761973 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":72,"threads_started":1}
I20260619 20:36:47.762145 679 tablet_service.cc:1467] Processing CreateTablet for tablet d4a763019f8e455b9667ce0e3b78fde2 (DEFAULT_TABLE table=test-table [id=4b23237e7713408cb20571d9181ab1ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:47.762269 679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4a763019f8e455b9667ce0e3b78fde2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:47.762748 831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.758399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:36761 (local address 127.0.0.1:48202)
0619 20:36:47.758691 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.758701 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:47.758741 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:47.759896 (+ 1155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:47.759897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:47.759899 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:47.759961 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:47.759962 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.760697 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.760698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:47.761850 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:47.761852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.762580 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.762581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:47.762624 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:47.762627 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:36:47.762675 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 20:36:47.762702 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":19,"thread_start_us":87,"threads_started":1}
I20260619 20:36:47.762799 833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.758583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48202 (local address 127.25.62.130:36761)
0619 20:36:47.759733 (+ 1150us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:47.759735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:47.759742 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:36:47.759768 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:47.759769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:47.759781 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:47.759840 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:47.760034 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.760559 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.761942 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.762497 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.762571 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:47.762581 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:47.762696 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 20:36:47.762732 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1129,"thread_start_us":59,"threads_started":1}
I20260619 20:36:47.763101 543 tablet_service.cc:1467] Processing CreateTablet for tablet d4a763019f8e455b9667ce0e3b78fde2 (DEFAULT_TABLE table=test-table [id=4b23237e7713408cb20571d9181ab1ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:47.763222 543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4a763019f8e455b9667ce0e3b78fde2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:47.763648 834 tablet_bootstrap.cc:492] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936: Bootstrap starting.
I20260619 20:36:47.763907 406 tablet_service.cc:1467] Processing CreateTablet for tablet d4a763019f8e455b9667ce0e3b78fde2 (DEFAULT_TABLE table=test-table [id=4b23237e7713408cb20571d9181ab1ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:47.764025 406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4a763019f8e455b9667ce0e3b78fde2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:47.764027 834 tablet_bootstrap.cc:654] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:47.764479 836 tablet_bootstrap.cc:492] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e: Bootstrap starting.
I20260619 20:36:47.764842 836 tablet_bootstrap.cc:654] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:47.765440 836 tablet_bootstrap.cc:492] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e: No bootstrap required, opened a new log
I20260619 20:36:47.765501 836 ts_tablet_manager.cc:1397] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:47.765570 838 tablet_bootstrap.cc:492] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164: Bootstrap starting.
I20260619 20:36:47.765611 836 raft_consensus.cc:348] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } } peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } }
I20260619 20:36:47.765692 836 raft_consensus.cc:374] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:47.765718 836 raft_consensus.cc:729] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a576e1317b724daeaea1183f19e0548e, State: Initialized, Role: FOLLOWER
I20260619 20:36:47.765784 836 consensus_queue.cc:260] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } } peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } }
I20260619 20:36:47.765902 838 tablet_bootstrap.cc:654] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:47.765945 836 ts_tablet_manager.cc:1428] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:47.766181 834 tablet_bootstrap.cc:492] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936: No bootstrap required, opened a new log
I20260619 20:36:47.766243 834 ts_tablet_manager.cc:1397] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 20:36:47.766389 841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.766190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.130:53463)
0619 20:36:47.766324 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.766359 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":74,"threads_started":1}
I20260619 20:36:47.766390 834 raft_consensus.cc:348] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } } peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } }
I20260619 20:36:47.766459 834 raft_consensus.cc:374] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:47.766487 834 raft_consensus.cc:729] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7589a1cd5a8d467cbd7d9610a2db3936, State: Initialized, Role: FOLLOWER
I20260619 20:36:47.766503 838 tablet_bootstrap.cc:492] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164: No bootstrap required, opened a new log
I20260619 20:36:47.766532 834 consensus_queue.cc:260] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } } peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } }
I20260619 20:36:47.766553 838 ts_tablet_manager.cc:1397] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:47.766682 834 ts_tablet_manager.cc:1428] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:47.766657 838 raft_consensus.cc:348] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } } peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } }
I20260619 20:36:47.766724 838 raft_consensus.cc:374] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:47.766754 838 raft_consensus.cc:729] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afca6bfafe0342f6a6e7167ae9632164, State: Initialized, Role: FOLLOWER
I20260619 20:36:47.766813 838 consensus_queue.cc:260] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } } peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } }
I20260619 20:36:47.767022 838 ts_tablet_manager.cc:1428] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:47.767403 844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.766979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.131:35379)
0619 20:36:47.767317 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.767351 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":70,"threads_started":1}
I20260619 20:36:47.767601 609 heartbeater.cc:503] Master 127.25.62.188:45945 was elected leader, sending a full tablet report...
I20260619 20:36:47.767933 745 heartbeater.cc:503] Master 127.25.62.188:45945 was elected leader, sending a full tablet report...
I20260619 20:36:47.767942 845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.767642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.129:44323)
0619 20:36:47.767873 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.767910 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":78,"threads_started":1}
I20260619 20:36:47.768445 841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.767221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.130:56849)
0619 20:36:47.768386 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.768418 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1095}
I20260619 20:36:47.768584 472 heartbeater.cc:503] Master 127.25.62.188:45945 was elected leader, sending a full tablet report...
I20260619 20:36:47.768875 845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.768801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.129:33513)
0619 20:36:47.768831 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.768854 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260619 20:36:47.773475 475 tablet.cc:2369] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:36:47.794976 748 tablet.cc:2369] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:36:47.880363 843 raft_consensus.cc:482] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:36:47.880465 843 raft_consensus.cc:504] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } } peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } }
I20260619 20:36:47.880780 843 leader_election.cc:288] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7589a1cd5a8d467cbd7d9610a2db3936 (127.25.62.131:43649), a576e1317b724daeaea1183f19e0548e (127.25.62.130:36761)
I20260619 20:36:47.883898 830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.881060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:48795 (local address 127.25.62.131:43649)
0619 20:36:47.881110 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:47.881112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:47.881121 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:36:47.881233 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:47.881234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:47.881248 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:47.881319 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:47.881492 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.882000 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.883143 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.883563 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.883619 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:47.883627 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:47.883784 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 20:36:47.883830 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:36:47.883889 845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.880890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:43649 (local address 127.25.62.129:48795)
0619 20:36:47.880951 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.880961 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:47.881033 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:47.881334 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:47.881335 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:47.881339 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:47.881421 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:47.881423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.882134 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.882135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:47.883057 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:47.883060 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.883637 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.883638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:47.883710 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:47.883714 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:47.883766 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 20:36:47.883796 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 20:36:47.883968 833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.881028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:53037 (local address 127.25.62.130:36761)
0619 20:36:47.881099 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:47.881101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:47.881319 (+ 218us) server_negotiation.cc:407] Connection header received
0619 20:36:47.881367 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:47.881368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:47.881377 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:47.881430 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:47.881573 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.882177 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.883172 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.883582 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.883686 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:47.883697 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:47.883846 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 20:36:47.883884 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 20:36:47.884052 846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.880931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:36761 (local address 127.25.62.129:53037)
0619 20:36:47.881235 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.881247 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:47.881312 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:47.881440 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:47.881441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:47.881443 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:47.881506 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:47.881508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.882302 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.882303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:47.883067 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:47.883069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.883683 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.883684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:47.883768 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:47.883773 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:36:47.883830 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 20:36:47.883902 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":130,"threads_started":1}
I20260619 20:36:47.884109 699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4a763019f8e455b9667ce0e3b78fde2" candidate_uuid: "afca6bfafe0342f6a6e7167ae9632164" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" is_pre_election: true
I20260619 20:36:47.884191 699 raft_consensus.cc:2454] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afca6bfafe0342f6a6e7167ae9632164 in term 0.
I20260619 20:36:47.884291 563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4a763019f8e455b9667ce0e3b78fde2" candidate_uuid: "afca6bfafe0342f6a6e7167ae9632164" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a576e1317b724daeaea1183f19e0548e" is_pre_election: true
I20260619 20:36:47.884472 563 raft_consensus.cc:2454] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afca6bfafe0342f6a6e7167ae9632164 in term 0.
I20260619 20:36:47.884662 359 leader_election.cc:302] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7589a1cd5a8d467cbd7d9610a2db3936, afca6bfafe0342f6a6e7167ae9632164; no voters:
I20260619 20:36:47.884792 843 raft_consensus.cc:2781] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:36:47.884838 843 raft_consensus.cc:482] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:36:47.884869 843 raft_consensus.cc:3037] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:47.885704 843 raft_consensus.cc:504] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } } peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } }
I20260619 20:36:47.885852 843 leader_election.cc:288] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [CANDIDATE]: Term 1 election: Requested vote from peers 7589a1cd5a8d467cbd7d9610a2db3936 (127.25.62.131:43649), a576e1317b724daeaea1183f19e0548e (127.25.62.130:36761)
I20260619 20:36:47.885982 563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4a763019f8e455b9667ce0e3b78fde2" candidate_uuid: "afca6bfafe0342f6a6e7167ae9632164" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a576e1317b724daeaea1183f19e0548e"
I20260619 20:36:47.886020 699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4a763019f8e455b9667ce0e3b78fde2" candidate_uuid: "afca6bfafe0342f6a6e7167ae9632164" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7589a1cd5a8d467cbd7d9610a2db3936"
I20260619 20:36:47.886070 563 raft_consensus.cc:3037] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:47.886106 699 raft_consensus.cc:3037] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:47.886719 563 raft_consensus.cc:2454] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afca6bfafe0342f6a6e7167ae9632164 in term 1.
I20260619 20:36:47.886718 699 raft_consensus.cc:2454] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afca6bfafe0342f6a6e7167ae9632164 in term 1.
I20260619 20:36:47.886920 359 leader_election.cc:302] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7589a1cd5a8d467cbd7d9610a2db3936, afca6bfafe0342f6a6e7167ae9632164; no voters:
I20260619 20:36:47.887028 843 raft_consensus.cc:2781] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:47.887162 843 raft_consensus.cc:686] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [term 1 LEADER]: Becoming Leader. State: Replica: afca6bfafe0342f6a6e7167ae9632164, State: Running, Role: LEADER
I20260619 20:36:47.887243 843 consensus_queue.cc:237] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } } peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } }
I20260619 20:36:47.887621 846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.887497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.129:40837)
0619 20:36:47.887547 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.887593 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 20:36:47.887990 32760 catalog_manager.cc:5495] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 reported cstate change: term changed from 0 to 1, leader changed from <none> to afca6bfafe0342f6a6e7167ae9632164 (127.25.62.129). New cstate: current_term: 1 leader_uuid: "afca6bfafe0342f6a6e7167ae9632164" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afca6bfafe0342f6a6e7167ae9632164" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 } health_report { overall_health: UNKNOWN } } }
I20260619 20:36:47.909199 832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.906285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37794 (local address 127.25.62.129:33507)
0619 20:36:47.906359 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:47.906361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:47.906374 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:36:47.906410 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:47.906411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:47.906436 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:47.906515 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:47.906693 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.907490 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.908322 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:47.908546 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:47.908616 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:47.908773 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:47.908898 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:47.908995 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:47.909088 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 20:36:47.909113 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 20:36:47.910423 699 raft_consensus.cc:1261] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936 [term 1 FOLLOWER]: Refusing update from remote peer afca6bfafe0342f6a6e7167ae9632164: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:47.910562 563 raft_consensus.cc:1261] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e [term 1 FOLLOWER]: Refusing update from remote peer afca6bfafe0342f6a6e7167ae9632164: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:47.910734 847 consensus_queue.cc:1035] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7589a1cd5a8d467cbd7d9610a2db3936" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 43649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:47.910956 843 consensus_queue.cc:1035] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a576e1317b724daeaea1183f19e0548e" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 36761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:47.911746 841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.911634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.130:60933)
0619 20:36:47.911683 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.911716 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 20:36:47.912420 844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.912297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.131:38261)
0619 20:36:47.912355 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.912388 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 20:36:47.913169 846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.912832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.129:39679)
0619 20:36:47.913108 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.913139 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260}
I20260619 20:36:47.913672 844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.913577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.131:33337)
0619 20:36:47.913618 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.913646 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 20:36:47.913991 851 mvcc.cc:205] Tried to move back new op lower bound from 7298668166798970880 to 7298668166706946048. Current Snapshot: MvccSnapshot[applied={T|T < 7298668166798970880 or (T in {7298668166798970880})}]
I20260619 20:36:47.914093 841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.914002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.130:58085)
0619 20:36:47.914038 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.914069 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 20:36:47.914353 846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.914236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42409 (local address 127.25.62.129:55953)
0619 20:36:47.914299 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.914328 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:42409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 20:36:47.915697 32760 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:47.915788 32760 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:47.917028 32760 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 694e1bf8ae0c4f94a8e26b6fff3c31f3: Sending DeleteTablet for 3 replicas of tablet d4a763019f8e455b9667ce0e3b78fde2
I20260619 20:36:47.917348 679 tablet_service.cc:1514] Processing DeleteTablet for tablet d4a763019f8e455b9667ce0e3b78fde2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:47 UTC) from {username='oryx', principal='oryx/127.25.62.188@KRBTEST.COM'} at 127.0.0.1:49324
I20260619 20:36:47.917371 543 tablet_service.cc:1514] Processing DeleteTablet for tablet d4a763019f8e455b9667ce0e3b78fde2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:47 UTC) from {username='oryx', principal='oryx/127.25.62.188@KRBTEST.COM'} at 127.0.0.1:48202
I20260619 20:36:47.917531 406 tablet_service.cc:1514] Processing DeleteTablet for tablet d4a763019f8e455b9667ce0e3b78fde2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:47 UTC) from {username='oryx', principal='oryx/127.25.62.188@KRBTEST.COM'} at 127.0.0.1:37786
I20260619 20:36:47.917572 860 tablet_replica.cc:330] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936: stopping tablet replica
I20260619 20:36:47.917618 860 raft_consensus.cc:2229] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:47.917673 860 raft_consensus.cc:2258] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:47.917770 859 tablet_replica.cc:330] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e: stopping tablet replica
I20260619 20:36:47.917825 859 raft_consensus.cc:2229] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:47.917773 861 tablet_replica.cc:330] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164: stopping tablet replica
I20260619 20:36:47.917876 859 raft_consensus.cc:2258] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:47.917953 861 raft_consensus.cc:2229] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:36:47.918056 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 32717
I20260619 20:36:47.918068 861 raft_consensus.cc:2258] T d4a763019f8e455b9667ce0e3b78fde2 P afca6bfafe0342f6a6e7167ae9632164 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:47.918634 860 ts_tablet_manager.cc:1905] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:47.920535 860 ts_tablet_manager.cc:1918] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:36:47.920598 860 log.cc:1196] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/d4a763019f8e455b9667ce0e3b78fde2
I20260619 20:36:47.920840 860 ts_tablet_manager.cc:1939] T d4a763019f8e455b9667ce0e3b78fde2 P 7589a1cd5a8d467cbd7d9610a2db3936: Deleting consensus metadata
I20260619 20:36:47.921264 859 ts_tablet_manager.cc:1905] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:47.921277 32745 catalog_manager.cc:4841] TS 7589a1cd5a8d467cbd7d9610a2db3936 (127.25.62.131:43649): tablet d4a763019f8e455b9667ce0e3b78fde2 (table test-table [id=4b23237e7713408cb20571d9181ab1ce]) successfully deleted
I20260619 20:36:47.923152 859 ts_tablet_manager.cc:1918] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:36:47.923213 859 log.cc:1196] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/d4a763019f8e455b9667ce0e3b78fde2
I20260619 20:36:47.923451 859 ts_tablet_manager.cc:1939] T d4a763019f8e455b9667ce0e3b78fde2 P a576e1317b724daeaea1183f19e0548e: Deleting consensus metadata
I20260619 20:36:47.923743 32746 catalog_manager.cc:4841] TS a576e1317b724daeaea1183f19e0548e (127.25.62.130:36761): tablet d4a763019f8e455b9667ce0e3b78fde2 (table test-table [id=4b23237e7713408cb20571d9181ab1ce]) successfully deleted
I20260619 20:36:47.924901 831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:47.924810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:33507 (local address 127.0.0.1:37798)
0619 20:36:47.924854 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:47.924876 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.129:33507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260619 20:36:47.925019 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 480
W20260619 20:36:47.925081 32748 catalog_manager.cc:4584] TS afca6bfafe0342f6a6e7167ae9632164 (127.25.62.129:33507): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d4a763019f8e455b9667ce0e3b78fde2: Network error: Client connection negotiation failed: client connection to 127.25.62.129:33507: connect: Connection refused (error 111)
I20260619 20:36:47.930930 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 616
I20260619 20:36:47.935979 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 32435
I20260619 20:36:47.940821 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 32622
2026-06-19T20:36:47Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4621 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:48 dist-test-slave-0t40 krb5kdc[864](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:48 dist-test-slave-0t40 krb5kdc[864](info): set up 2 sockets
Jun 19 20:36:48 dist-test-slave-0t40 krb5kdc[864](info): commencing operation
krb5kdc: starting...
W20260619 20:36:48.985865 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:36:49 dist-test-slave-0t40 krb5kdc[864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:49Z Disabled control of system clock
WARNING: no policy specified for oryx/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.190.keytab.
Entry for principal oryx/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.190.keytab.
I20260619 20:36:49.103811 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:42717
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.190.keytab
--principal=oryx/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:42717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:49.182205 877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:49.182446 877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:49.182479 877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:49.183621 877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:49.183691 877 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 20:36:49.183713 877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:49.183736 877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:49.183765 877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:49.183789 877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:49.185360 877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:42717
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.190.keytab
--principal=oryx/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:42717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:49.185643 877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:49.185863 877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:49.190477 883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:49.190498 885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:49.190514 882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:49.190810 877 server_base.cc:1034] running on GCE node
I20260619 20:36:49.191119 877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:49.191413 877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:49.192584 877 hybrid_clock.cc:648] HybridClock initialized: now 1781901409192555 us; error 37 us; skew 500 ppm
Jun 19 20:36:49 dist-test-slave-0t40 krb5kdc[864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901409, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:49.195045 877 init.cc:377] Logged in from keytab as oryx/127.25.62.190@KRBTEST.COM (short username oryx)
I20260619 20:36:49.195482 877 webserver.cc:460] Webserver started at http://127.25.62.190:36469/ using document root <none> and password file <none>
I20260619 20:36:49.195713 877 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:49.195780 877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:49.195933 877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:49.196974 877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "556c0f4af6ee4aeaaacd88eed02f0653"
format_stamp: "Formatted at 2026-06-19 20:36:49 on dist-test-slave-0t40"
server_key: "c9c80d07d85dd13fea92d96a41409858"
server_key_iv: "63c5470a5586c0160a449b207a984743"
server_key_version: "encryptionkey@0"
I20260619 20:36:49.197397 877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "556c0f4af6ee4aeaaacd88eed02f0653"
format_stamp: "Formatted at 2026-06-19 20:36:49 on dist-test-slave-0t40"
server_key: "c9c80d07d85dd13fea92d96a41409858"
server_key_iv: "63c5470a5586c0160a449b207a984743"
server_key_version: "encryptionkey@0"
I20260619 20:36:49.198701 877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:49.199453 892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:49.199630 877 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:49.199723 877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "556c0f4af6ee4aeaaacd88eed02f0653"
format_stamp: "Formatted at 2026-06-19 20:36:49 on dist-test-slave-0t40"
server_key: "c9c80d07d85dd13fea92d96a41409858"
server_key_iv: "63c5470a5586c0160a449b207a984743"
server_key_version: "encryptionkey@0"
I20260619 20:36:49.199810 877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:49.207954 877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:49.210273 877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:49.210441 877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:49.215200 877 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:42717
I20260619 20:36:49.215288 954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:42717 every 8 connection(s)
I20260619 20:36:49.215600 877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:49.216244 955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:49.218482 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 877
I20260619 20:36:49.218587 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:36:49.218614 955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653: Bootstrap starting.
I20260619 20:36:49.218776 25850 external_mini_cluster.cc:1428] Setting key e3e2272df277fb15c0b8f3406b6ab272
I20260619 20:36:49.219372 955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:49.219667 955 log.cc:824] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:49.220386 955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653: No bootstrap required, opened a new log
I20260619 20:36:49.222731 955 raft_consensus.cc:348] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "556c0f4af6ee4aeaaacd88eed02f0653" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42717 } }
I20260619 20:36:49.222913 955 raft_consensus.cc:374] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:49.222994 955 raft_consensus.cc:729] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 556c0f4af6ee4aeaaacd88eed02f0653, State: Initialized, Role: FOLLOWER
I20260619 20:36:49.223130 955 consensus_queue.cc:260] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "556c0f4af6ee4aeaaacd88eed02f0653" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42717 } }
I20260619 20:36:49.223227 955 raft_consensus.cc:388] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:49.223268 955 raft_consensus.cc:482] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:49.223315 955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:49.223966 955 raft_consensus.cc:504] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "556c0f4af6ee4aeaaacd88eed02f0653" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42717 } }
I20260619 20:36:49.224087 955 leader_election.cc:302] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 556c0f4af6ee4aeaaacd88eed02f0653; no voters:
I20260619 20:36:49.224251 955 leader_election.cc:288] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:49.224299 960 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:49.224466 960 raft_consensus.cc:686] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653 [term 1 LEADER]: Becoming Leader. State: Replica: 556c0f4af6ee4aeaaacd88eed02f0653, State: Running, Role: LEADER
I20260619 20:36:49.224506 955 sys_catalog.cc:564] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:49.224575 960 consensus_queue.cc:237] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "556c0f4af6ee4aeaaacd88eed02f0653" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42717 } }
I20260619 20:36:49.224895 960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 556c0f4af6ee4aeaaacd88eed02f0653. Latest consensus state: current_term: 1 leader_uuid: "556c0f4af6ee4aeaaacd88eed02f0653" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "556c0f4af6ee4aeaaacd88eed02f0653" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42717 } } }
I20260619 20:36:49.224982 960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:49.224908 962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "556c0f4af6ee4aeaaacd88eed02f0653" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "556c0f4af6ee4aeaaacd88eed02f0653" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42717 } } }
I20260619 20:36:49.225108 962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653 [sys.catalog]: This master's current role is: LEADER
Jun 19 20:36:49 dist-test-slave-0t40 krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM
W20260619 20:36:49.226532 976 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:36:49.226598 976 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:36:49.226655 965 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:49.226816 965 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:49.228219 965 catalog_manager.cc:1347] Generated new cluster ID: 4f4f24f2067c43678a85cf941bef8c30
I20260619 20:36:49.228271 965 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:49.230222 958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.219266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56584 (local address 127.25.62.190:42717)
0619 20:36:49.219439 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:49.219449 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:49.219483 (+ 34us) server_negotiation.cc:407] Connection header received
0619 20:36:49.220026 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:49.220033 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:49.220083 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:49.220199 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:49.220962 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.221845 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.222925 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.223512 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.223593 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.226462 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:49.226467 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:49.226467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:49.226479 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:49.228532 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:49.228907 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:49.228910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:49.228910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:49.229014 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:49.229306 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:49.229307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:49.229307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:49.229613 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:49.229763 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:49.229870 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 20:36:49.230011 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260619 20:36:49.239820 965 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:49.240340 965 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:49.242695 965 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653: Generated new TSK 0
I20260619 20:36:49.242866 965 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.129.keytab.
Entry for principal oryx/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.129.keytab.
I20260619 20:36:49.274246 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:42717
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.129.keytab
--principal=oryx/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:49.355566 981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:49.355769 981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:49.355795 981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:49.356869 981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:49.356911 981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:49.356935 981 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:36:49.356954 981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:36:49.358467 981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.129.keytab
--principal=oryx/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.62.190:42717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:49.358647 981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:49.358816 981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:49.363665 986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:49.363665 987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:49.363691 989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:49.364180 981 server_base.cc:1034] running on GCE node
I20260619 20:36:49.364425 981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:49.364678 981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:49.365818 981 hybrid_clock.cc:648] HybridClock initialized: now 1781901409365806 us; error 30 us; skew 500 ppm
Jun 19 20:36:49 dist-test-slave-0t40 krb5kdc[864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901409, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:49.368324 981 init.cc:377] Logged in from keytab as oryx/127.25.62.129@KRBTEST.COM (short username oryx)
I20260619 20:36:49.368693 981 webserver.cc:460] Webserver started at http://127.25.62.129:45509/ using document root <none> and password file <none>
I20260619 20:36:49.368865 981 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:49.368903 981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:49.368999 981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:49.370010 981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "31d910f6497f450dbecbd5cd3e6ab642"
format_stamp: "Formatted at 2026-06-19 20:36:49 on dist-test-slave-0t40"
server_key: "e1ee36208118705990e8cb6bc491134e"
server_key_iv: "720b55f255478b483960106dd48515bd"
server_key_version: "encryptionkey@0"
I20260619 20:36:49.370298 981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "31d910f6497f450dbecbd5cd3e6ab642"
format_stamp: "Formatted at 2026-06-19 20:36:49 on dist-test-slave-0t40"
server_key: "e1ee36208118705990e8cb6bc491134e"
server_key_iv: "720b55f255478b483960106dd48515bd"
server_key_version: "encryptionkey@0"
I20260619 20:36:49.371506 981 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:36:49.372201 996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:49.372399 981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:49.372496 981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "31d910f6497f450dbecbd5cd3e6ab642"
format_stamp: "Formatted at 2026-06-19 20:36:49 on dist-test-slave-0t40"
server_key: "e1ee36208118705990e8cb6bc491134e"
server_key_iv: "720b55f255478b483960106dd48515bd"
server_key_version: "encryptionkey@0"
I20260619 20:36:49.372578 981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:49.381146 981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:49.383373 981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:49.383533 981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:49.384019 981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:49.384083 981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:49.384125 981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:49.384150 981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:49.390918 981 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:43649
Jun 19 20:36:49 dist-test-slave-0t40 krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901409, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM
I20260619 20:36:49.390987 1111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:43649 every 8 connection(s)
I20260619 20:36:49.391297 981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:36:49.394006 958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.384521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:52485 (local address 127.25.62.190:42717)
0619 20:36:49.384592 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:49.384594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:49.384797 (+ 203us) server_negotiation.cc:407] Connection header received
0619 20:36:49.385349 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:49.385351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:49.385365 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:49.385454 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:49.386275 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.387147 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.388712 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.389061 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.389151 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.391509 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:49.391511 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:49.391512 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:49.391520 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:49.392874 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:49.393299 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:49.393300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:49.393300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:49.393350 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:49.393545 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:49.393545 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:49.393545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:49.393634 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:49.393679 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:49.393875 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 20:36:49.393937 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 20:36:49.394230 1005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.384498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42717 (local address 127.25.62.129:52485)
0619 20:36:49.384666 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:49.384705 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:49.385250 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:49.385507 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:49.385510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:49.385871 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:49.386167 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.386174 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.387248 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.387249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.388619 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.388623 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.389162 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.389163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.389259 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.389791 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:49.389798 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:49.391398 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:49.392940 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:49.392957 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:49.392959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:49.393236 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:49.393408 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:49.393409 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:49.393409 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:49.393502 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:49.393683 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:49.393687 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:49.393789 (+ 102us) client_negotiation.cc:769] Sending connection context
0619 20:36:49.393866 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 20:36:49.393989 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260619 20:36:49.394716 1112 heartbeater.cc:347] Connected to a master server at 127.25.62.190:42717
I20260619 20:36:49.394814 1112 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:49.394990 1112 heartbeater.cc:511] Master 127.25.62.190:42717 requested a full tablet report, sending...
I20260619 20:36:49.395506 908 ts_manager.cc:194] Registered new tserver with Master: 31d910f6497f450dbecbd5cd3e6ab642 (127.25.62.129:43649)
I20260619 20:36:49.396450 908 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:52485
I20260619 20:36:49.398736 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 981
I20260619 20:36:49.398845 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:36:49.399044 25850 external_mini_cluster.cc:1428] Setting key cbc41c0aab325a73bac2e141eebb3964
I20260619 20:36:49.401510 958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.395678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:57435 (local address 127.25.62.190:42717)
0619 20:36:49.395719 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:49.395720 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:49.395853 (+ 133us) server_negotiation.cc:407] Connection header received
0619 20:36:49.395922 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:49.395923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:49.395931 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:49.395977 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:49.396304 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.396821 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.397621 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.397809 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.397890 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.398792 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:49.398793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:49.398793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:49.398800 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:49.400400 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:49.400719 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:49.400720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:49.400720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:49.400777 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:49.400955 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:49.400956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:49.400956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:49.401107 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:49.401179 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:49.401312 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 20:36:49.401391 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:36:49.401526 1118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.395634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42717 (local address 127.25.62.129:57435)
0619 20:36:49.395797 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:49.395808 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:49.395869 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:49.395998 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:49.395999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:49.396175 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:49.396238 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.396240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.396914 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.396915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.397525 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.397528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.397892 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.397893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.397957 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.398372 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:49.398377 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:49.398686 (+ 309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:49.400483 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:49.400483 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:49.400484 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:49.400650 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:49.400839 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:49.400840 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:49.400840 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:49.400889 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:49.401169 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:49.401169 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:49.401214 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 20:36:49.401266 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 20:36:49.401376 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":77,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for oryx/127.25.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.130.keytab.
Entry for principal oryx/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.130.keytab.
I20260619 20:36:49.425299 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:42717
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.130.keytab
--principal=oryx/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:49.505607 1121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:49.505808 1121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:49.505832 1121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:49.506997 1121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:49.507041 1121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:49.507062 1121 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:36:49.507081 1121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:36:49.508615 1121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.130.keytab
--principal=oryx/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.62.190:42717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:49.508882 1121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:49.509145 1121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:49.514292 1126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:49.514302 1127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:49.514302 1129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:49.514495 1121 server_base.cc:1034] running on GCE node
I20260619 20:36:49.514712 1121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:49.514971 1121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:49.516132 1121 hybrid_clock.cc:648] HybridClock initialized: now 1781901409516128 us; error 36 us; skew 500 ppm
Jun 19 20:36:49 dist-test-slave-0t40 krb5kdc[864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901409, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:49.518641 1121 init.cc:377] Logged in from keytab as oryx/127.25.62.130@KRBTEST.COM (short username oryx)
I20260619 20:36:49.519069 1121 webserver.cc:460] Webserver started at http://127.25.62.130:40895/ using document root <none> and password file <none>
I20260619 20:36:49.519311 1121 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:49.519356 1121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:49.519508 1121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:49.520542 1121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "954d742a8afe4e35b466850bfeef94d1"
format_stamp: "Formatted at 2026-06-19 20:36:49 on dist-test-slave-0t40"
server_key: "677586e555e1252f3c1c6b01444a1ccc"
server_key_iv: "4613c730deb95132d54f997d7f8dfc8d"
server_key_version: "encryptionkey@0"
I20260619 20:36:49.520885 1121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "954d742a8afe4e35b466850bfeef94d1"
format_stamp: "Formatted at 2026-06-19 20:36:49 on dist-test-slave-0t40"
server_key: "677586e555e1252f3c1c6b01444a1ccc"
server_key_iv: "4613c730deb95132d54f997d7f8dfc8d"
server_key_version: "encryptionkey@0"
I20260619 20:36:49.522260 1121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:49.523067 1136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:49.523245 1121 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:49.523343 1121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "954d742a8afe4e35b466850bfeef94d1"
format_stamp: "Formatted at 2026-06-19 20:36:49 on dist-test-slave-0t40"
server_key: "677586e555e1252f3c1c6b01444a1ccc"
server_key_iv: "4613c730deb95132d54f997d7f8dfc8d"
server_key_version: "encryptionkey@0"
I20260619 20:36:49.523427 1121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:49.531335 1121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:49.533245 1121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:49.533396 1121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:49.533861 1121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:49.533931 1121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:49.533972 1121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:49.534000 1121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:49.540901 1121 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:46379
I20260619 20:36:49.541004 1251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:46379 every 8 connection(s)
I20260619 20:36:49.541296 1121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:36:49 dist-test-slave-0t40 krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901409, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM
I20260619 20:36:49.544591 958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.534352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:38193 (local address 127.25.62.190:42717)
0619 20:36:49.534421 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:49.534423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:49.534689 (+ 266us) server_negotiation.cc:407] Connection header received
0619 20:36:49.535253 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:49.535254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:49.535267 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:49.535332 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:49.536268 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.537146 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.538831 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.539074 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.539131 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.541991 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:49.541992 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:49.541992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:49.542000 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:49.543191 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:49.543609 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:49.543610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:49.543611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:49.543677 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:49.543948 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:49.543949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:49.543949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:49.544070 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:49.544149 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:49.544417 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 20:36:49.544497 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:36:49.544790 1145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.534333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42717 (local address 127.25.62.130:38193)
0619 20:36:49.534566 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:49.534594 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:49.535154 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:49.535444 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:49.535448 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:49.535851 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:49.536150 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.536158 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.537237 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.537239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.538727 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.538731 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.539169 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.539171 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.539327 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.540007 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:49.540018 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:49.541868 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:49.543277 (+ 1409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:49.543284 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:49.543286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:49.543511 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:49.543746 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:49.543746 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:49.543747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:49.543867 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:49.544139 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:49.544146 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:49.544286 (+ 140us) client_negotiation.cc:769] Sending connection context
0619 20:36:49.544397 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 20:36:49.544556 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":87,"threads_started":1}
I20260619 20:36:49.545331 1252 heartbeater.cc:347] Connected to a master server at 127.25.62.190:42717
I20260619 20:36:49.545404 1252 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:49.545593 1252 heartbeater.cc:511] Master 127.25.62.190:42717 requested a full tablet report, sending...
I20260619 20:36:49.545958 909 ts_manager.cc:194] Registered new tserver with Master: 954d742a8afe4e35b466850bfeef94d1 (127.25.62.130:46379)
I20260619 20:36:49.546460 909 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:38193
I20260619 20:36:49.550019 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 1121
I20260619 20:36:49.550143 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:36:49.550361 25850 external_mini_cluster.cc:1428] Setting key 4d5faccf7fcb0f051636412b6e6036e6
I20260619 20:36:49.551968 1258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.546258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42717 (local address 127.25.62.130:48581)
0619 20:36:49.546432 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:49.546443 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:49.546508 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:49.546633 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:49.546634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:49.546784 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:49.546840 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.546841 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.547492 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.547493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.548022 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.548024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.548400 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.548401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.548465 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.548869 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:49.548872 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:49.549212 (+ 340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:49.550950 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:49.550951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:49.550951 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:49.551155 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:49.551351 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:49.551352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:49.551352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:49.551402 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:49.551730 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:49.551731 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:49.551766 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 20:36:49.551815 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 20:36:49.551888 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":99,"threads_started":1}
I20260619 20:36:49.552057 958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.546338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:48581 (local address 127.25.62.190:42717)
0619 20:36:49.546377 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:49.546379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:49.546506 (+ 127us) server_negotiation.cc:407] Connection header received
0619 20:36:49.546553 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:49.546554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:49.546565 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:49.546616 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:49.546923 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.547396 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.548112 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.548315 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.548398 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.549296 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:49.549297 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:49.549298 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:49.549305 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:49.550825 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:49.551226 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:49.551227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:49.551227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:49.551286 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:49.551462 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:49.551463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:49.551464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:49.551666 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:49.551738 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:49.551841 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 20:36:49.551925 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for oryx/127.25.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.131.keytab.
Entry for principal oryx/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.131.keytab.
I20260619 20:36:49.575762 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:42717
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.131.keytab
--principal=oryx/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:49.658281 1261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:49.658521 1261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:49.658565 1261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:49.659798 1261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:49.659871 1261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:49.659911 1261 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 20:36:49.659967 1261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:36:49.661579 1261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.131.keytab
--principal=oryx/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.25.62.190:42717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:49.661844 1261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:49.662088 1261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:49.667335 1267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:49.667336 1269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:49.667368 1266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:49.667352 1261 server_base.cc:1034] running on GCE node
I20260619 20:36:49.667678 1261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:49.667958 1261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:49.669128 1261 hybrid_clock.cc:648] HybridClock initialized: now 1781901409669119 us; error 37 us; skew 500 ppm
Jun 19 20:36:49 dist-test-slave-0t40 krb5kdc[864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901409, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:49.676461 1261 init.cc:377] Logged in from keytab as oryx/127.25.62.131@KRBTEST.COM (short username oryx)
I20260619 20:36:49.676947 1261 webserver.cc:460] Webserver started at http://127.25.62.131:46851/ using document root <none> and password file <none>
I20260619 20:36:49.677212 1261 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:49.677260 1261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:49.677358 1261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:49.678354 1261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "f8095c17dbb84f11acbdf8c1090822f3"
format_stamp: "Formatted at 2026-06-19 20:36:49 on dist-test-slave-0t40"
server_key: "dc92b8983280c01fd320654c32014d1c"
server_key_iv: "20b2b105c108b549f83336a2a73b3a68"
server_key_version: "encryptionkey@0"
I20260619 20:36:49.678682 1261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "f8095c17dbb84f11acbdf8c1090822f3"
format_stamp: "Formatted at 2026-06-19 20:36:49 on dist-test-slave-0t40"
server_key: "dc92b8983280c01fd320654c32014d1c"
server_key_iv: "20b2b105c108b549f83336a2a73b3a68"
server_key_version: "encryptionkey@0"
I20260619 20:36:49.679915 1261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:36:49.680614 1276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:49.680779 1261 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:49.680871 1261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "f8095c17dbb84f11acbdf8c1090822f3"
format_stamp: "Formatted at 2026-06-19 20:36:49 on dist-test-slave-0t40"
server_key: "dc92b8983280c01fd320654c32014d1c"
server_key_iv: "20b2b105c108b549f83336a2a73b3a68"
server_key_version: "encryptionkey@0"
I20260619 20:36:49.680932 1261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:49.689740 1261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:49.692106 1261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:49.692225 1261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:49.692693 1261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:49.692741 1261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:49.692766 1261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:49.692824 1261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 20:36:49 dist-test-slave-0t40 krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901409, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM
I20260619 20:36:49.700750 1261 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:44601
I20260619 20:36:49.700829 1391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:44601 every 8 connection(s)
I20260619 20:36:49.701179 1261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:36:49.703670 958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.693216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:58195 (local address 127.25.62.190:42717)
0619 20:36:49.693263 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:49.693266 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:49.693573 (+ 307us) server_negotiation.cc:407] Connection header received
0619 20:36:49.694330 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:49.694331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:49.694343 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:49.694432 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:49.695347 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.696171 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.697969 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.698299 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.698394 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.700890 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:49.700892 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:49.700892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:49.700900 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:49.702360 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:49.702710 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:49.702711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:49.702711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:49.702760 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:49.703019 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:49.703020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:49.703021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:49.703145 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:49.703213 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:49.703498 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 20:36:49.703582 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:36:49.703858 1285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.693235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42717 (local address 127.25.62.131:58195)
0619 20:36:49.693400 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:49.693455 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:49.694215 (+ 760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:49.694515 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:49.694520 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:49.694940 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:49.695242 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.695248 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.696273 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.696274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.697870 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.697874 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.698413 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.698414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.698581 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.699071 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:49.699082 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:49.700803 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:49.702430 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:49.702436 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:49.702437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:49.702645 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:49.702826 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:49.702826 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:49.702827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:49.702938 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:49.703221 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:49.703225 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:49.703331 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 20:36:49.703467 (+ 136us) client_negotiation.cc:240] Negotiation successful
0619 20:36:49.703618 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
I20260619 20:36:49.704355 1392 heartbeater.cc:347] Connected to a master server at 127.25.62.190:42717
I20260619 20:36:49.704435 1392 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:49.704618 1392 heartbeater.cc:511] Master 127.25.62.190:42717 requested a full tablet report, sending...
I20260619 20:36:49.705078 908 ts_manager.cc:194] Registered new tserver with Master: f8095c17dbb84f11acbdf8c1090822f3 (127.25.62.131:44601)
I20260619 20:36:49.705557 908 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:58195
I20260619 20:36:49.710260 958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.705516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:35605 (local address 127.25.62.190:42717)
0619 20:36:49.705552 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:49.705554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:49.705670 (+ 116us) server_negotiation.cc:407] Connection header received
0619 20:36:49.705809 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:49.705810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:49.705820 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:49.705872 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:49.706154 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.706662 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.707302 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.707477 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.707524 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.708232 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:49.708234 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:49.708234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:49.708239 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:49.709460 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:49.709741 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:49.709742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:49.709742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:49.709786 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:49.709936 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:49.709936 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:49.709936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:49.710014 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:49.710057 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:49.710159 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 20:36:49.710208 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:36:49.710278 1398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.705463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42717 (local address 127.25.62.131:35605)
0619 20:36:49.705591 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:49.705601 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:49.705748 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:49.705874 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:49.705875 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:49.706022 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:49.706079 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.706080 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.706744 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.706745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.707238 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.707239 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.707535 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.707536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.707581 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.707907 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:49.707911 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:49.708158 (+ 247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:49.709523 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:49.709524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:49.709524 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:49.709679 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:49.709840 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:49.709840 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:49.709841 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:49.709888 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:49.710061 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:49.710061 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:49.710094 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 20:36:49.710143 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 20:36:49.710211 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260619 20:36:49.710764 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 1261
I20260619 20:36:49.710875 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:36:49.711102 25850 external_mini_cluster.cc:1428] Setting key f6b892b218aaea35f90a4f66182b6736
I20260619 20:36:49.712024 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 20:36:49 dist-test-slave-0t40 krb5kdc[864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901409, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 20:36:49 dist-test-slave-0t40 krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:49 dist-test-slave-0t40 krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 20:36:49.733553 958 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:49.728025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56594 (local address 127.25.62.190:42717)
0619 20:36:49.728081 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:49.728084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:49.728092 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:49.728162 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:49.728163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:49.728180 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:49.728245 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:49.728838 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.729409 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.730145 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.730330 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.730406 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.733496 (+ 3090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56594: BlockingRecv error: recv got EOF from 127.0.0.1:56594 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 19 20:36:49 dist-test-slave-0t40 krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901409, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM
I20260619 20:36:49.741245 958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.734559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56602 (local address 127.25.62.190:42717)
0619 20:36:49.734614 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:49.734615 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:49.734721 (+ 106us) server_negotiation.cc:407] Connection header received
0619 20:36:49.734748 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:49.734749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:49.734759 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:49.734815 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:49.735247 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.735710 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.736409 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.736591 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.736653 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.738827 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:49.738828 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:49.738828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:49.738834 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:49.740266 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:49.740580 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:49.740580 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:49.740580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:49.740628 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:49.740784 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:49.740785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:49.740785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:49.740876 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:49.740959 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:49.741107 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 20:36:49.741163 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 20:36:49.742882 908 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:36:49.744483 908 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:36:49.750921 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.746843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:46379 (local address 127.0.0.1:52676)
0619 20:36:49.747145 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:49.747189 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:49.747265 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:49.747464 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:49.747467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:49.747470 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:49.747532 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.747533 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.748821 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.748822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.749835 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.749837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.750596 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.750597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.750686 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.750700 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 20:36:49.750775 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 20:36:49.750830 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":98,"threads_started":1}
I20260619 20:36:49.750938 1416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.746928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52676 (local address 127.25.62.130:46379)
0619 20:36:49.747095 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:49.747100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:49.747244 (+ 144us) server_negotiation.cc:407] Connection header received
0619 20:36:49.747345 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:49.747347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:49.747362 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:49.747457 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:49.747630 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.748719 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.749912 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.750507 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.750580 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.750651 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:49.750799 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 20:36:49.750839 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260619 20:36:49.751325 1414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.747027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:43649 (local address 127.0.0.1:47332)
0619 20:36:49.747577 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:49.747586 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:49.747631 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:49.747879 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:49.747880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:49.747882 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:49.747939 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.747940 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.749155 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.749156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.750205 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.750208 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.751101 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.751102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.751171 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.751175 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:49.751229 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 20:36:49.751264 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":69,"threads_started":1}
I20260619 20:36:49.751375 1419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.747239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47332 (local address 127.25.62.129:43649)
0619 20:36:49.747393 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:49.747398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:49.747689 (+ 291us) server_negotiation.cc:407] Connection header received
0619 20:36:49.747731 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:49.747733 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:49.747748 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:49.747816 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:49.748022 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.749074 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.750288 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.751011 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.751088 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.751160 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:49.751238 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 20:36:49.751278 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260619 20:36:49.751570 1415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.746843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:44601 (local address 127.0.0.1:54614)
0619 20:36:49.747413 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:49.747423 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:49.747484 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:49.747728 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:49.747729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:49.747731 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:49.747786 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.747787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.748655 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.748656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.749849 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.749851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.750777 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.750778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.750837 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.750840 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:36:49.750885 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 20:36:49.750929 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":4,"thread_start_us":43,"threads_started":1}
I20260619 20:36:49.751639 1168 tablet_service.cc:1467] Processing CreateTablet for tablet c454e529ff6246e09aed59b1979f1675 (DEFAULT_TABLE table=test-table [id=d7245233992344c6b7c8e1f00c3e291e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:49.751832 1418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.747052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54614 (local address 127.25.62.131:44601)
0619 20:36:49.747208 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:49.747212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:49.747477 (+ 265us) server_negotiation.cc:407] Connection header received
0619 20:36:49.747563 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:49.747566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:49.747582 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:49.747658 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:49.747860 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.748567 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.749956 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.750606 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.750671 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.750744 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:49.750970 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 20:36:49.751714 (+ 744us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260619 20:36:49.752070 1168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c454e529ff6246e09aed59b1979f1675. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:49.752410 1326 tablet_service.cc:1467] Processing CreateTablet for tablet c454e529ff6246e09aed59b1979f1675 (DEFAULT_TABLE table=test-table [id=d7245233992344c6b7c8e1f00c3e291e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:49.752705 1326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c454e529ff6246e09aed59b1979f1675. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:49.753789 1046 tablet_service.cc:1467] Processing CreateTablet for tablet c454e529ff6246e09aed59b1979f1675 (DEFAULT_TABLE table=test-table [id=d7245233992344c6b7c8e1f00c3e291e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:49.754117 1046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c454e529ff6246e09aed59b1979f1675. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:49.755733 1420 tablet_bootstrap.cc:492] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3: Bootstrap starting.
I20260619 20:36:49.755985 1421 tablet_bootstrap.cc:492] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1: Bootstrap starting.
I20260619 20:36:49.756422 1420 tablet_bootstrap.cc:654] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:49.756609 1421 tablet_bootstrap.cc:654] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:49.756765 1420 log.cc:824] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:49.756984 1421 log.cc:824] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:49.757707 1421 tablet_bootstrap.cc:492] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1: No bootstrap required, opened a new log
I20260619 20:36:49.757707 1420 tablet_bootstrap.cc:492] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3: No bootstrap required, opened a new log
I20260619 20:36:49.757788 1420 ts_tablet_manager.cc:1397] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:36:49.757788 1421 ts_tablet_manager.cc:1397] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 20:36:49.757995 1424 tablet_bootstrap.cc:492] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642: Bootstrap starting.
I20260619 20:36:49.758486 1424 tablet_bootstrap.cc:654] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:49.758771 1424 log.cc:824] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:49.759413 1424 tablet_bootstrap.cc:492] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642: No bootstrap required, opened a new log
I20260619 20:36:49.759497 1424 ts_tablet_manager.cc:1397] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:36:49.759932 1421 raft_consensus.cc:348] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } } peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } }
I20260619 20:36:49.760129 1421 raft_consensus.cc:374] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:49.760169 1421 raft_consensus.cc:729] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 954d742a8afe4e35b466850bfeef94d1, State: Initialized, Role: FOLLOWER
I20260619 20:36:49.760075 1420 raft_consensus.cc:348] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } } peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } }
I20260619 20:36:49.760214 1420 raft_consensus.cc:374] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:49.760246 1420 raft_consensus.cc:729] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8095c17dbb84f11acbdf8c1090822f3, State: Initialized, Role: FOLLOWER
I20260619 20:36:49.760293 1421 consensus_queue.cc:260] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } } peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } }
I20260619 20:36:49.760375 1420 consensus_queue.cc:260] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } } peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } }
I20260619 20:36:49.760598 1421 ts_tablet_manager.cc:1428] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 20:36:49.760622 1420 ts_tablet_manager.cc:1428] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:36:49.760720 1392 heartbeater.cc:503] Master 127.25.62.190:42717 was elected leader, sending a full tablet report...
I20260619 20:36:49.761010 1252 heartbeater.cc:503] Master 127.25.62.190:42717 was elected leader, sending a full tablet report...
I20260619 20:36:49.761694 1424 raft_consensus.cc:348] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } } peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } }
I20260619 20:36:49.761828 1424 raft_consensus.cc:374] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:49.761850 1424 raft_consensus.cc:729] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31d910f6497f450dbecbd5cd3e6ab642, State: Initialized, Role: FOLLOWER
I20260619 20:36:49.761942 1424 consensus_queue.cc:260] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } } peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } }
I20260619 20:36:49.762188 1424 ts_tablet_manager.cc:1428] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:36:49.762212 1112 heartbeater.cc:503] Master 127.25.62.190:42717 was elected leader, sending a full tablet report...
I20260619 20:36:49.764278 1428 raft_consensus.cc:482] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:36:49.764357 1428 raft_consensus.cc:504] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } } peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } }
I20260619 20:36:49.764854 1428 leader_election.cc:288] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 954d742a8afe4e35b466850bfeef94d1 (127.25.62.130:46379), f8095c17dbb84f11acbdf8c1090822f3 (127.25.62.131:44601)
I20260619 20:36:49.767452 1416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.764953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:41837 (local address 127.25.62.130:46379)
0619 20:36:49.764993 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:49.764995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:49.765004 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:36:49.765026 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:49.765027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:49.765058 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:49.765131 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:49.765267 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.765784 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.766731 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.767096 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.767151 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.767157 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:49.767345 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 20:36:49.767384 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:36:49.767472 1418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.764942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:44081 (local address 127.25.62.131:44601)
0619 20:36:49.764983 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:49.764985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:49.765139 (+ 154us) server_negotiation.cc:407] Connection header received
0619 20:36:49.765201 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:49.765202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:49.765210 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:49.765268 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:49.765382 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.765838 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.766732 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.767111 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.767202 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.767211 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:49.767367 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 20:36:49.767403 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:36:49.767508 1429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.764838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:44601 (local address 127.25.62.129:44081)
0619 20:36:49.765089 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:49.765099 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:49.765149 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:49.765267 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:49.765268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:49.765269 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:49.765331 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.765332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.765920 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.765921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.766638 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.766640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.767210 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.767211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.767281 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.767284 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:36:49.767353 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 20:36:49.767395 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":43,"thread_start_us":78,"threads_started":1}
I20260619 20:36:49.767525 1005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.764838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:46379 (local address 127.25.62.129:41837)
0619 20:36:49.764892 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:49.764904 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:49.764948 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:49.765136 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:49.765137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:49.765139 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:49.765215 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.765217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.765862 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.765863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.766630 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.766633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.767171 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.767172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.767267 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.767271 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:49.767323 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 20:36:49.767390 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":2}
I20260619 20:36:49.767815 1190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c454e529ff6246e09aed59b1979f1675" candidate_uuid: "31d910f6497f450dbecbd5cd3e6ab642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "954d742a8afe4e35b466850bfeef94d1" is_pre_election: true
I20260619 20:36:49.767975 1190 raft_consensus.cc:2454] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31d910f6497f450dbecbd5cd3e6ab642 in term 0.
I20260619 20:36:49.767939 1346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c454e529ff6246e09aed59b1979f1675" candidate_uuid: "31d910f6497f450dbecbd5cd3e6ab642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8095c17dbb84f11acbdf8c1090822f3" is_pre_election: true
I20260619 20:36:49.768054 1346 raft_consensus.cc:2454] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31d910f6497f450dbecbd5cd3e6ab642 in term 0.
I20260619 20:36:49.768214 998 leader_election.cc:302] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31d910f6497f450dbecbd5cd3e6ab642, 954d742a8afe4e35b466850bfeef94d1; no voters:
I20260619 20:36:49.768342 1428 raft_consensus.cc:2781] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:36:49.768400 1428 raft_consensus.cc:482] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:36:49.768417 1428 raft_consensus.cc:3037] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:49.769461 1428 raft_consensus.cc:504] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } } peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } }
I20260619 20:36:49.769591 1428 leader_election.cc:288] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [CANDIDATE]: Term 1 election: Requested vote from peers 954d742a8afe4e35b466850bfeef94d1 (127.25.62.130:46379), f8095c17dbb84f11acbdf8c1090822f3 (127.25.62.131:44601)
I20260619 20:36:49.769727 1346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c454e529ff6246e09aed59b1979f1675" candidate_uuid: "31d910f6497f450dbecbd5cd3e6ab642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8095c17dbb84f11acbdf8c1090822f3"
I20260619 20:36:49.769769 1190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c454e529ff6246e09aed59b1979f1675" candidate_uuid: "31d910f6497f450dbecbd5cd3e6ab642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "954d742a8afe4e35b466850bfeef94d1"
I20260619 20:36:49.769835 1346 raft_consensus.cc:3037] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:49.769832 1190 raft_consensus.cc:3037] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:49.770488 1190 raft_consensus.cc:2454] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31d910f6497f450dbecbd5cd3e6ab642 in term 1.
I20260619 20:36:49.770618 1346 raft_consensus.cc:2454] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31d910f6497f450dbecbd5cd3e6ab642 in term 1.
I20260619 20:36:49.770648 998 leader_election.cc:302] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31d910f6497f450dbecbd5cd3e6ab642, 954d742a8afe4e35b466850bfeef94d1; no voters:
I20260619 20:36:49.770720 1428 raft_consensus.cc:2781] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:49.770855 1428 raft_consensus.cc:686] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 LEADER]: Becoming Leader. State: Replica: 31d910f6497f450dbecbd5cd3e6ab642, State: Running, Role: LEADER
I20260619 20:36:49.770978 1428 consensus_queue.cc:237] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } } peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } }
I20260619 20:36:49.771821 908 catalog_manager.cc:5495] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31d910f6497f450dbecbd5cd3e6ab642 (127.25.62.129). New cstate: current_term: 1 leader_uuid: "31d910f6497f450dbecbd5cd3e6ab642" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } health_report { overall_health: HEALTHY } } }
I20260619 20:36:49.778286 1419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.775778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47336 (local address 127.25.62.129:43649)
0619 20:36:49.775831 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:49.775834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:49.775842 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:49.775911 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:49.775912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:49.775928 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:49.776003 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:49.776149 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.776661 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.777507 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.777713 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.777796 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.777978 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:49.778075 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:49.778165 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:49.778190 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 20:36:49.778209 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 20:36:49.779781 1190 raft_consensus.cc:1261] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1 [term 1 FOLLOWER]: Refusing update from remote peer 31d910f6497f450dbecbd5cd3e6ab642: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:49.779804 1346 raft_consensus.cc:1261] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3 [term 1 FOLLOWER]: Refusing update from remote peer 31d910f6497f450dbecbd5cd3e6ab642: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:49.780085 1428 consensus_queue.cc:1035] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:49.780196 1430 consensus_queue.cc:1035] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:49.783185 1435 mvcc.cc:205] Tried to move back new op lower bound from 7298668174455742464 to 7298668174422663168. Current Snapshot: MvccSnapshot[applied={T|T < 7298668174455742464}]
I20260619 20:36:49.783129 1434 mvcc.cc:205] Tried to move back new op lower bound from 7298668174455742464 to 7298668174422663168. Current Snapshot: MvccSnapshot[applied={T|T < 7298668174455742464}]
I20260619 20:36:49.784544 909 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:49.784629 909 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:49.785394 909 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653: Sending DeleteTablet for 3 replicas of tablet c454e529ff6246e09aed59b1979f1675
I20260619 20:36:49.785705 1326 tablet_service.cc:1514] Processing DeleteTablet for tablet c454e529ff6246e09aed59b1979f1675 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:49 UTC) from {username='oryx', principal='oryx/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:54614
I20260619 20:36:49.785737 1168 tablet_service.cc:1514] Processing DeleteTablet for tablet c454e529ff6246e09aed59b1979f1675 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:49 UTC) from {username='oryx', principal='oryx/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:52676
I20260619 20:36:49.785883 909 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 20:36:49.785995 1046 tablet_service.cc:1514] Processing DeleteTablet for tablet c454e529ff6246e09aed59b1979f1675 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:49 UTC) from {username='oryx', principal='oryx/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:47332
W20260619 20:36:49.786028 909 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:36:49.786352 1444 tablet_replica.cc:330] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1: stopping tablet replica
I20260619 20:36:49.786465 1444 raft_consensus.cc:2229] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:49.786377 1443 tablet_replica.cc:330] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3: stopping tablet replica
I20260619 20:36:49.786751 1445 tablet_replica.cc:330] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642: stopping tablet replica
I20260619 20:36:49.786846 1445 raft_consensus.cc:2229] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:36:49.786947 1445 raft_consensus.cc:2258] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:49.787091 1444 raft_consensus.cc:2258] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:49.787408 1046 tablet_service.cc:1467] Processing CreateTablet for tablet cb5ad23662614e15b27a3b0ccee0004f (DEFAULT_TABLE table=test-table [id=f4ddceeb6d1042a38e2a3d7e5d9df224]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:49.787523 1046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb5ad23662614e15b27a3b0ccee0004f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:49.787528 1326 tablet_service.cc:1467] Processing CreateTablet for tablet cb5ad23662614e15b27a3b0ccee0004f (DEFAULT_TABLE table=test-table [id=f4ddceeb6d1042a38e2a3d7e5d9df224]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:49.787647 1326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb5ad23662614e15b27a3b0ccee0004f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:49.787873 1444 ts_tablet_manager.cc:1905] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:49.788811 1424 tablet_bootstrap.cc:492] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642: Bootstrap starting.
I20260619 20:36:49.788873 1168 tablet_service.cc:1467] Processing CreateTablet for tablet cb5ad23662614e15b27a3b0ccee0004f (DEFAULT_TABLE table=test-table [id=f4ddceeb6d1042a38e2a3d7e5d9df224]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:49.788985 1168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb5ad23662614e15b27a3b0ccee0004f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:49.789151 1420 tablet_bootstrap.cc:492] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3: Bootstrap starting.
I20260619 20:36:49.789263 1424 tablet_bootstrap.cc:654] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:49.787524 1445 ts_tablet_manager.cc:1905] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:49.789592 1420 tablet_bootstrap.cc:654] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:49.790419 1421 tablet_bootstrap.cc:492] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1: Bootstrap starting.
I20260619 20:36:49.790813 1421 tablet_bootstrap.cc:654] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:49.790810 1443 raft_consensus.cc:2229] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:49.791016 1443 raft_consensus.cc:2258] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:49.791092 1445 ts_tablet_manager.cc:1918] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:36:49.791163 1445 log.cc:1196] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/wal/wals/c454e529ff6246e09aed59b1979f1675
I20260619 20:36:49.791417 1445 ts_tablet_manager.cc:1939] T c454e529ff6246e09aed59b1979f1675 P 31d910f6497f450dbecbd5cd3e6ab642: Deleting consensus metadata
I20260619 20:36:49.791570 1443 ts_tablet_manager.cc:1905] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:49.791834 894 catalog_manager.cc:4841] TS 31d910f6497f450dbecbd5cd3e6ab642 (127.25.62.129:43649): tablet c454e529ff6246e09aed59b1979f1675 (table test-table [id=d7245233992344c6b7c8e1f00c3e291e]) successfully deleted
I20260619 20:36:49.792136 1444 ts_tablet_manager.cc:1918] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:36:49.792192 1444 log.cc:1196] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/c454e529ff6246e09aed59b1979f1675
I20260619 20:36:49.792363 1444 ts_tablet_manager.cc:1939] T c454e529ff6246e09aed59b1979f1675 P 954d742a8afe4e35b466850bfeef94d1: Deleting consensus metadata
I20260619 20:36:49.792933 894 catalog_manager.cc:4841] TS 954d742a8afe4e35b466850bfeef94d1 (127.25.62.130:46379): tablet c454e529ff6246e09aed59b1979f1675 (table test-table [id=d7245233992344c6b7c8e1f00c3e291e]) successfully deleted
I20260619 20:36:49.793028 1424 tablet_bootstrap.cc:492] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642: No bootstrap required, opened a new log
I20260619 20:36:49.793103 1424 ts_tablet_manager.cc:1397] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260619 20:36:49.793262 1424 raft_consensus.cc:348] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:49.793342 1424 raft_consensus.cc:374] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:49.793365 1424 raft_consensus.cc:729] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31d910f6497f450dbecbd5cd3e6ab642, State: Initialized, Role: FOLLOWER
I20260619 20:36:49.793363 1420 tablet_bootstrap.cc:492] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3: No bootstrap required, opened a new log
I20260619 20:36:49.793427 1424 consensus_queue.cc:260] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:49.793449 1420 ts_tablet_manager.cc:1397] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 20:36:49.793524 1424 ts_tablet_manager.cc:1428] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:49.793618 1420 raft_consensus.cc:348] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:49.793687 1420 raft_consensus.cc:374] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:49.793718 1420 raft_consensus.cc:729] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8095c17dbb84f11acbdf8c1090822f3, State: Initialized, Role: FOLLOWER
I20260619 20:36:49.793778 1420 consensus_queue.cc:260] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:49.793865 1420 ts_tablet_manager.cc:1428] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:49.794214 1443 ts_tablet_manager.cc:1918] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:36:49.794235 1421 tablet_bootstrap.cc:492] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1: No bootstrap required, opened a new log
I20260619 20:36:49.794277 1421 ts_tablet_manager.cc:1397] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260619 20:36:49.794371 1443 log.cc:1196] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/c454e529ff6246e09aed59b1979f1675
I20260619 20:36:49.794445 1421 raft_consensus.cc:348] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:49.794530 1421 raft_consensus.cc:374] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:49.794560 1421 raft_consensus.cc:729] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 954d742a8afe4e35b466850bfeef94d1, State: Initialized, Role: FOLLOWER
I20260619 20:36:49.794631 1421 consensus_queue.cc:260] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:49.794740 1421 ts_tablet_manager.cc:1428] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 20:36:49.794785 1443 ts_tablet_manager.cc:1939] T c454e529ff6246e09aed59b1979f1675 P f8095c17dbb84f11acbdf8c1090822f3: Deleting consensus metadata
I20260619 20:36:49.795179 896 catalog_manager.cc:4841] TS f8095c17dbb84f11acbdf8c1090822f3 (127.25.62.131:44601): tablet c454e529ff6246e09aed59b1979f1675 (table test-table [id=d7245233992344c6b7c8e1f00c3e291e]) successfully deleted
I20260619 20:36:49.889158 1430 raft_consensus.cc:482] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:36:49.889271 1430 raft_consensus.cc:504] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:49.889451 1430 leader_election.cc:288] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 954d742a8afe4e35b466850bfeef94d1 (127.25.62.130:46379), f8095c17dbb84f11acbdf8c1090822f3 (127.25.62.131:44601)
I20260619 20:36:49.889653 1346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5ad23662614e15b27a3b0ccee0004f" candidate_uuid: "31d910f6497f450dbecbd5cd3e6ab642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8095c17dbb84f11acbdf8c1090822f3" is_pre_election: true
I20260619 20:36:49.889739 1346 raft_consensus.cc:2454] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31d910f6497f450dbecbd5cd3e6ab642 in term 0.
I20260619 20:36:49.889671 1190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5ad23662614e15b27a3b0ccee0004f" candidate_uuid: "31d910f6497f450dbecbd5cd3e6ab642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "954d742a8afe4e35b466850bfeef94d1" is_pre_election: true
I20260619 20:36:49.889801 1190 raft_consensus.cc:2454] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31d910f6497f450dbecbd5cd3e6ab642 in term 0.
I20260619 20:36:49.889899 1000 leader_election.cc:302] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31d910f6497f450dbecbd5cd3e6ab642, f8095c17dbb84f11acbdf8c1090822f3; no voters:
I20260619 20:36:49.889997 1430 raft_consensus.cc:2781] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:36:49.890033 1430 raft_consensus.cc:482] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:36:49.890064 1430 raft_consensus.cc:3037] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:49.890712 1430 raft_consensus.cc:504] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:49.890852 1430 leader_election.cc:288] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [CANDIDATE]: Term 1 election: Requested vote from peers 954d742a8afe4e35b466850bfeef94d1 (127.25.62.130:46379), f8095c17dbb84f11acbdf8c1090822f3 (127.25.62.131:44601)
I20260619 20:36:49.891043 1190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5ad23662614e15b27a3b0ccee0004f" candidate_uuid: "31d910f6497f450dbecbd5cd3e6ab642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "954d742a8afe4e35b466850bfeef94d1"
I20260619 20:36:49.891065 1346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb5ad23662614e15b27a3b0ccee0004f" candidate_uuid: "31d910f6497f450dbecbd5cd3e6ab642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8095c17dbb84f11acbdf8c1090822f3"
I20260619 20:36:49.891139 1190 raft_consensus.cc:3037] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:49.891155 1346 raft_consensus.cc:3037] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:49.891753 1346 raft_consensus.cc:2454] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31d910f6497f450dbecbd5cd3e6ab642 in term 1.
I20260619 20:36:49.891917 1000 leader_election.cc:302] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31d910f6497f450dbecbd5cd3e6ab642, f8095c17dbb84f11acbdf8c1090822f3; no voters:
I20260619 20:36:49.891989 1190 raft_consensus.cc:2454] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31d910f6497f450dbecbd5cd3e6ab642 in term 1.
W20260619 20:36:49.892067 1113 tablet.cc:2369] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:36:49.892099 1430 raft_consensus.cc:2781] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:49.892138 1430 raft_consensus.cc:686] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 LEADER]: Becoming Leader. State: Replica: 31d910f6497f450dbecbd5cd3e6ab642, State: Running, Role: LEADER
I20260619 20:36:49.892177 1430 consensus_queue.cc:237] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:49.892767 909 catalog_manager.cc:5495] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31d910f6497f450dbecbd5cd3e6ab642 (127.25.62.129). New cstate: current_term: 1 leader_uuid: "31d910f6497f450dbecbd5cd3e6ab642" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } health_report { overall_health: UNKNOWN } } }
Jun 19 20:36:49 dist-test-slave-0t40 krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901409, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.190@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM
I20260619 20:36:49.936340 958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.929767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56612 (local address 127.25.62.190:42717)
0619 20:36:49.929851 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:49.929853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:49.929938 (+ 85us) server_negotiation.cc:407] Connection header received
0619 20:36:49.929984 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:49.929985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:49.929994 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:49.930046 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:49.930483 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.930971 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.931711 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.931894 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.931948 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.934035 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:49.934036 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:49.934036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:49.934042 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:49.935428 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:49.935753 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:49.935754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:49.935754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:49.935802 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:49.935965 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:49.935966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:49.935966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:49.936059 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:49.936108 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:49.936218 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 20:36:49.936278 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 20:36:49.936503 1455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:49.929693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42717 (local address 127.0.0.1:56612)
0619 20:36:49.929851 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:49.929860 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:49.929941 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:49.930066 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:49.930067 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:49.930363 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:49.930414 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.930416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.931104 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.931105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.931634 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:49.931635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:49.931960 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:49.931960 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:49.932000 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:49.932498 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:49.932502 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:49.933957 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:49.935501 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:49.935507 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:49.935509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:49.935691 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:49.935856 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:49.935857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:49.935857 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:49.935915 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:49.936111 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:49.936116 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:49.936153 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 20:36:49.936204 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 20:36:49.936338 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":53,"thread_start_us":75,"threads_started":1}
I20260619 20:36:49.938383 909 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:56612:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 20:36:49.938587 909 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:36:49.939997 1046 tablet_service.cc:1467] Processing CreateTablet for tablet 812dfc2a90ad41048c5f8b1f3cf25c88 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a486b5bdae4f40edb771bbe3d36511be]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:36:49.940049 1326 tablet_service.cc:1467] Processing CreateTablet for tablet 812dfc2a90ad41048c5f8b1f3cf25c88 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a486b5bdae4f40edb771bbe3d36511be]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:36:49.940166 1326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 812dfc2a90ad41048c5f8b1f3cf25c88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:49.940167 1046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 812dfc2a90ad41048c5f8b1f3cf25c88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:49.941196 1168 tablet_service.cc:1467] Processing CreateTablet for tablet 812dfc2a90ad41048c5f8b1f3cf25c88 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a486b5bdae4f40edb771bbe3d36511be]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 20:36:49.941347 1168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 812dfc2a90ad41048c5f8b1f3cf25c88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:49.941679 1424 tablet_bootstrap.cc:492] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 31d910f6497f450dbecbd5cd3e6ab642: Bootstrap starting.
I20260619 20:36:49.942034 1424 tablet_bootstrap.cc:654] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 31d910f6497f450dbecbd5cd3e6ab642: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:49.942297 1420 tablet_bootstrap.cc:492] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3: Bootstrap starting.
I20260619 20:36:49.942716 1420 tablet_bootstrap.cc:654] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:49.942936 1421 tablet_bootstrap.cc:492] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1: Bootstrap starting.
I20260619 20:36:49.943290 1424 tablet_bootstrap.cc:492] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 31d910f6497f450dbecbd5cd3e6ab642: No bootstrap required, opened a new log
I20260619 20:36:49.943333 1424 ts_tablet_manager.cc:1397] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 31d910f6497f450dbecbd5cd3e6ab642: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 20:36:49.943369 1421 tablet_bootstrap.cc:654] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:49.943466 1424 raft_consensus.cc:348] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:49.943535 1424 raft_consensus.cc:374] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:49.943568 1424 raft_consensus.cc:729] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31d910f6497f450dbecbd5cd3e6ab642, State: Initialized, Role: FOLLOWER
I20260619 20:36:49.943616 1424 consensus_queue.cc:260] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 31d910f6497f450dbecbd5cd3e6ab642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:49.943728 1424 ts_tablet_manager.cc:1428] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 31d910f6497f450dbecbd5cd3e6ab642: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 20:36:49.943750 1430 tablet_replica.cc:439] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 31d910f6497f450dbecbd5cd3e6ab642: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } } }
I20260619 20:36:49.943847 1430 tablet_replica.cc:442] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 31d910f6497f450dbecbd5cd3e6ab642: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:49.944144 1420 tablet_bootstrap.cc:492] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3: No bootstrap required, opened a new log
I20260619 20:36:49.944192 1420 ts_tablet_manager.cc:1397] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:36:49.944324 1421 tablet_bootstrap.cc:492] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1: No bootstrap required, opened a new log
I20260619 20:36:49.944327 1420 raft_consensus.cc:348] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:49.944365 1421 ts_tablet_manager.cc:1397] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:49.944389 1420 raft_consensus.cc:374] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:49.944409 1420 raft_consensus.cc:729] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8095c17dbb84f11acbdf8c1090822f3, State: Initialized, Role: FOLLOWER
I20260619 20:36:49.944482 1420 consensus_queue.cc:260] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:49.944507 1421 raft_consensus.cc:348] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:49.944551 1421 raft_consensus.cc:374] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:49.944567 1420 ts_tablet_manager.cc:1428] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:49.944586 1421 raft_consensus.cc:729] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 954d742a8afe4e35b466850bfeef94d1, State: Initialized, Role: FOLLOWER
I20260619 20:36:49.944648 1421 consensus_queue.cc:260] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:49.944640 1427 tablet_replica.cc:439] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } } }
I20260619 20:36:49.944741 1421 ts_tablet_manager.cc:1428] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 20:36:49.944738 1427 tablet_replica.cc:442] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:49.944794 1426 tablet_replica.cc:439] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } } }
I20260619 20:36:49.944885 1426 tablet_replica.cc:442] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1: This TxnStatusTablet replica's current role is: FOLLOWER
W20260619 20:36:49.952039 1393 tablet.cc:2369] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:36:50.042115 1253 tablet.cc:2369] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:36:50.075225 1426 raft_consensus.cc:482] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:36:50.075320 1426 raft_consensus.cc:504] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.075711 1426 leader_election.cc:288] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31d910f6497f450dbecbd5cd3e6ab642 (127.25.62.129:43649), f8095c17dbb84f11acbdf8c1090822f3 (127.25.62.131:44601)
I20260619 20:36:50.078774 1418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:50.076055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:35857 (local address 127.25.62.131:44601)
0619 20:36:50.076108 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:50.076111 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:50.076177 (+ 66us) server_negotiation.cc:407] Connection header received
0619 20:36:50.076263 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:50.076264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:50.076276 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:50.076346 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:50.076511 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.076990 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.078063 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.078417 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.078511 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:50.078520 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:50.078666 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 20:36:50.078699 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 20:36:50.078790 1419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:50.076025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:47997 (local address 127.25.62.129:43649)
0619 20:36:50.076116 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:50.076119 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:50.076155 (+ 36us) server_negotiation.cc:407] Connection header received
0619 20:36:50.076240 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:50.076245 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:50.076258 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:50.076319 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:36:50.076513 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.076990 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.078076 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.078434 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.078520 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:50.078530 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:50.078681 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 20:36:50.078717 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:36:50.078827 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:50.075946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:44601 (local address 127.25.62.130:35857)
0619 20:36:50.076101 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:50.076113 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:50.076192 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:50.076349 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:50.076350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:50.076352 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:50.076427 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:50.076429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.077108 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.077109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:50.077975 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:50.077977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.078512 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.078513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:50.078591 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:50.078595 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:36:50.078661 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 20:36:50.078699 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":57,"thread_start_us":72,"threads_started":1}
I20260619 20:36:50.078836 1459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:50.075946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:43649 (local address 127.25.62.130:47997)
0619 20:36:50.076083 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:50.076100 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:50.076169 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:50.076336 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:50.076337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:50.076339 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:36:50.076424 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:50.076426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.077096 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.077097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:50.077974 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:50.077977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.078520 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.078521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:50.078597 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:50.078607 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 20:36:50.078676 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 20:36:50.078716 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":66,"thread_start_us":65,"threads_started":1}
I20260619 20:36:50.079162 1346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "812dfc2a90ad41048c5f8b1f3cf25c88" candidate_uuid: "954d742a8afe4e35b466850bfeef94d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8095c17dbb84f11acbdf8c1090822f3" is_pre_election: true
I20260619 20:36:50.079284 1346 raft_consensus.cc:2454] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 954d742a8afe4e35b466850bfeef94d1 in term 0.
I20260619 20:36:50.079317 1066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "812dfc2a90ad41048c5f8b1f3cf25c88" candidate_uuid: "954d742a8afe4e35b466850bfeef94d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31d910f6497f450dbecbd5cd3e6ab642" is_pre_election: true
I20260619 20:36:50.079458 1140 leader_election.cc:302] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 954d742a8afe4e35b466850bfeef94d1, f8095c17dbb84f11acbdf8c1090822f3; no voters:
I20260619 20:36:50.079478 1066 raft_consensus.cc:2454] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 954d742a8afe4e35b466850bfeef94d1 in term 0.
I20260619 20:36:50.079600 1426 raft_consensus.cc:2781] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:36:50.079641 1426 raft_consensus.cc:482] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:36:50.079667 1426 raft_consensus.cc:3037] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:50.080363 1426 raft_consensus.cc:504] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.080514 1426 leader_election.cc:288] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 [CANDIDATE]: Term 1 election: Requested vote from peers 31d910f6497f450dbecbd5cd3e6ab642 (127.25.62.129:43649), f8095c17dbb84f11acbdf8c1090822f3 (127.25.62.131:44601)
I20260619 20:36:50.080647 1346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "812dfc2a90ad41048c5f8b1f3cf25c88" candidate_uuid: "954d742a8afe4e35b466850bfeef94d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8095c17dbb84f11acbdf8c1090822f3"
I20260619 20:36:50.080646 1066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "812dfc2a90ad41048c5f8b1f3cf25c88" candidate_uuid: "954d742a8afe4e35b466850bfeef94d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31d910f6497f450dbecbd5cd3e6ab642"
I20260619 20:36:50.080724 1346 raft_consensus.cc:3037] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:50.080734 1066 raft_consensus.cc:3037] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:50.081385 1346 raft_consensus.cc:2454] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 954d742a8afe4e35b466850bfeef94d1 in term 1.
I20260619 20:36:50.081535 1066 raft_consensus.cc:2454] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 954d742a8afe4e35b466850bfeef94d1 in term 1.
I20260619 20:36:50.081545 1140 leader_election.cc:302] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 954d742a8afe4e35b466850bfeef94d1, f8095c17dbb84f11acbdf8c1090822f3; no voters:
I20260619 20:36:50.081634 1426 raft_consensus.cc:2781] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:50.081732 1426 raft_consensus.cc:686] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 [term 1 LEADER]: Becoming Leader. State: Replica: 954d742a8afe4e35b466850bfeef94d1, State: Running, Role: LEADER
I20260619 20:36:50.081825 1426 consensus_queue.cc:237] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.082166 1461 tablet_replica.cc:439] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1: TxnStatusTablet state changed. Reason: New leader 954d742a8afe4e35b466850bfeef94d1. Latest consensus state: current_term: 1 leader_uuid: "954d742a8afe4e35b466850bfeef94d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } } }
I20260619 20:36:50.082276 1461 tablet_replica.cc:442] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:36:50.082455 1463 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 20:36:50.082597 1463 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 20:36:50.082546 908 catalog_manager.cc:5495] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 954d742a8afe4e35b466850bfeef94d1 (127.25.62.130). New cstate: current_term: 1 leader_uuid: "954d742a8afe4e35b466850bfeef94d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } health_report { overall_health: UNKNOWN } } }
I20260619 20:36:50.089546 1416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:50.086936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52690 (local address 127.25.62.130:46379)
0619 20:36:50.086973 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:50.086975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:50.087006 (+ 31us) server_negotiation.cc:407] Connection header received
0619 20:36:50.087095 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:50.087096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:50.087109 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:50.087165 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:50.087279 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.087720 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.088774 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.088961 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.089014 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:50.089243 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:50.089327 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:50.089378 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:50.089465 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 20:36:50.089482 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:36:50.089591 1455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:50.086875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:46379 (local address 127.0.0.1:52690)
0619 20:36:50.086960 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:50.086971 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:50.087024 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:50.087169 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:50.087170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:50.087171 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:36:50.087220 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:50.087222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.087794 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.087795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:50.088694 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:50.088697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.089060 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.089061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:50.089131 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:50.089148 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:36:50.089389 (+ 241us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:36:50.089396 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 20:36:50.089463 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 20:36:50.089507 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 20:36:50.090564 1346 raft_consensus.cc:1261] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3 [term 1 FOLLOWER]: Refusing update from remote peer 954d742a8afe4e35b466850bfeef94d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:50.090807 1066 raft_consensus.cc:1261] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 FOLLOWER]: Refusing update from remote peer 954d742a8afe4e35b466850bfeef94d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:50.090849 1461 consensus_queue.cc:1035] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:50.091053 1426 consensus_queue.cc:1035] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:50.091666 1427 tablet_replica.cc:439] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3: TxnStatusTablet state changed. Reason: New leader 954d742a8afe4e35b466850bfeef94d1. Latest consensus state: current_term: 1 leader_uuid: "954d742a8afe4e35b466850bfeef94d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } } }
I20260619 20:36:50.091687 1430 tablet_replica.cc:439] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 31d910f6497f450dbecbd5cd3e6ab642: TxnStatusTablet state changed. Reason: New leader 954d742a8afe4e35b466850bfeef94d1. Latest consensus state: current_term: 1 leader_uuid: "954d742a8afe4e35b466850bfeef94d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } } }
I20260619 20:36:50.091758 1427 tablet_replica.cc:442] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:50.091758 1430 tablet_replica.cc:442] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 31d910f6497f450dbecbd5cd3e6ab642: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:50.092446 1461 tablet_replica.cc:439] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "954d742a8afe4e35b466850bfeef94d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } } }
I20260619 20:36:50.092531 1461 tablet_replica.cc:442] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:36:50.092592 1461 tablet_replica.cc:439] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "954d742a8afe4e35b466850bfeef94d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } } }
I20260619 20:36:50.092664 1461 tablet_replica.cc:442] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 954d742a8afe4e35b466850bfeef94d1: This TxnStatusTablet replica's current role is: LEADER
I20260619 20:36:50.092641 1430 tablet_replica.cc:439] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 31d910f6497f450dbecbd5cd3e6ab642: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "954d742a8afe4e35b466850bfeef94d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } } }
I20260619 20:36:50.092725 1430 tablet_replica.cc:442] T 812dfc2a90ad41048c5f8b1f3cf25c88 P 31d910f6497f450dbecbd5cd3e6ab642: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:50.092833 1427 tablet_replica.cc:439] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "954d742a8afe4e35b466850bfeef94d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } } }
I20260619 20:36:50.092912 1427 tablet_replica.cc:442] T 812dfc2a90ad41048c5f8b1f3cf25c88 P f8095c17dbb84f11acbdf8c1090822f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 20:36:50.093403 1433 mvcc.cc:205] Tried to move back new op lower bound from 7298668175729119232 to 7298668175695785984. Current Snapshot: MvccSnapshot[applied={T|T < 7298668175729119232 or (T in {7298668175729119232})}]
I20260619 20:36:50.099062 1416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:50.096423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:55097 (local address 127.25.62.130:46379)
0619 20:36:50.096464 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:50.096466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:50.096650 (+ 184us) server_negotiation.cc:407] Connection header received
0619 20:36:50.096704 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:50.096705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:50.096713 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:50.096758 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:50.096865 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.097392 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.098488 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.098655 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.098704 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:50.098875 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:50.098898 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:50.098945 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:50.099001 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 20:36:50.099016 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:36:50.099203 1470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:50.096388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:46379 (local address 127.25.62.129:55097)
0619 20:36:50.096568 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:50.096580 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:50.096648 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:50.096761 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:50.096762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:50.096764 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:36:50.096813 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:50.096814 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.097466 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.097467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:50.098399 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:50.098402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.098724 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.098725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:50.098785 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:50.098800 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:36:50.098950 (+ 150us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:36:50.098951 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 20:36:50.098999 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 20:36:50.099042 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":99,"threads_started":1}
I20260619 20:36:50.101234 1346 raft_consensus.cc:1261] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3 [term 1 FOLLOWER]: Refusing update from remote peer 31d910f6497f450dbecbd5cd3e6ab642: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:50.101266 1190 raft_consensus.cc:1261] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1 [term 1 FOLLOWER]: Refusing update from remote peer 31d910f6497f450dbecbd5cd3e6ab642: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:50.101459 1440 consensus_queue.cc:1035] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:50.101541 1430 consensus_queue.cc:1035] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:50.109218 1419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:50.106918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:39217 (local address 127.25.62.129:43649)
0619 20:36:50.106965 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:50.106967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:50.106991 (+ 24us) server_negotiation.cc:407] Connection header received
0619 20:36:50.107053 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:50.107054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:50.107063 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:50.107118 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:50.107270 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.107725 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.108516 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.108719 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.108800 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:50.108954 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:50.108986 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:50.109080 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:50.109134 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 20:36:50.109153 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 20:36:50.109262 1476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:50.106756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:43649 (local address 127.25.62.130:39217)
0619 20:36:50.106931 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:50.106947 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:50.107006 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:50.107121 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:50.107121 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:50.107123 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 20:36:50.107195 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:50.107196 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.107802 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.107803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:50.108455 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:50.108458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.108815 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.108816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:50.108874 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:50.108887 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 20:36:50.109074 (+ 187us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 20:36:50.109076 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:36:50.109130 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 20:36:50.109168 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
I20260619 20:36:50.118433 908 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:50.118572 908 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:50.119450 908 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653: Sending DeleteTablet for 3 replicas of tablet cb5ad23662614e15b27a3b0ccee0004f
I20260619 20:36:50.119792 1326 tablet_service.cc:1514] Processing DeleteTablet for tablet cb5ad23662614e15b27a3b0ccee0004f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:50 UTC) from {username='oryx', principal='oryx/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:54614
I20260619 20:36:50.119794 1168 tablet_service.cc:1514] Processing DeleteTablet for tablet cb5ad23662614e15b27a3b0ccee0004f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:50 UTC) from {username='oryx', principal='oryx/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:52676
I20260619 20:36:50.119930 1444 tablet_replica.cc:330] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1: stopping tablet replica
I20260619 20:36:50.120012 1444 raft_consensus.cc:2229] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:50.120008 1443 tablet_replica.cc:330] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3: stopping tablet replica
I20260619 20:36:50.120069 1444 raft_consensus.cc:2258] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:50.120097 1443 raft_consensus.cc:2229] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:50.120159 1443 raft_consensus.cc:2258] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:50.120404 1444 ts_tablet_manager.cc:1905] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:50.122690 1443 ts_tablet_manager.cc:1905] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:50.123606 1444 ts_tablet_manager.cc:1918] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:36:50.123693 1444 log.cc:1196] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/cb5ad23662614e15b27a3b0ccee0004f
I20260619 20:36:50.123924 1444 ts_tablet_manager.cc:1939] T cb5ad23662614e15b27a3b0ccee0004f P 954d742a8afe4e35b466850bfeef94d1: Deleting consensus metadata
I20260619 20:36:50.124301 1443 ts_tablet_manager.cc:1918] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:36:50.124331 894 catalog_manager.cc:4841] TS 954d742a8afe4e35b466850bfeef94d1 (127.25.62.130:46379): tablet cb5ad23662614e15b27a3b0ccee0004f (table test-table [id=f4ddceeb6d1042a38e2a3d7e5d9df224]) successfully deleted
I20260619 20:36:50.124372 1443 log.cc:1196] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/cb5ad23662614e15b27a3b0ccee0004f
I20260619 20:36:50.124610 1443 ts_tablet_manager.cc:1939] T cb5ad23662614e15b27a3b0ccee0004f P f8095c17dbb84f11acbdf8c1090822f3: Deleting consensus metadata
I20260619 20:36:50.125087 896 catalog_manager.cc:4841] TS f8095c17dbb84f11acbdf8c1090822f3 (127.25.62.131:44601): tablet cb5ad23662614e15b27a3b0ccee0004f (table test-table [id=f4ddceeb6d1042a38e2a3d7e5d9df224]) successfully deleted
I20260619 20:36:50.126492 1046 tablet_service.cc:1514] Processing DeleteTablet for tablet cb5ad23662614e15b27a3b0ccee0004f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:50 UTC) from {username='oryx', principal='oryx/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:47332
I20260619 20:36:50.126618 1445 tablet_replica.cc:330] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642: stopping tablet replica
I20260619 20:36:50.126698 1445 raft_consensus.cc:2229] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:36:50.126784 1445 raft_consensus.cc:2258] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:50.127058 1445 ts_tablet_manager.cc:1905] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:50.129002 1445 ts_tablet_manager.cc:1918] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:36:50.129103 1445 log.cc:1196] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/wal/wals/cb5ad23662614e15b27a3b0ccee0004f
I20260619 20:36:50.129348 1445 ts_tablet_manager.cc:1939] T cb5ad23662614e15b27a3b0ccee0004f P 31d910f6497f450dbecbd5cd3e6ab642: Deleting consensus metadata
I20260619 20:36:50.129733 894 catalog_manager.cc:4841] TS 31d910f6497f450dbecbd5cd3e6ab642 (127.25.62.129:43649): tablet cb5ad23662614e15b27a3b0ccee0004f (table test-table [id=f4ddceeb6d1042a38e2a3d7e5d9df224]) successfully deleted
Jun 19 20:36:50 dist-test-slave-0t40 krb5kdc[864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 20:36:50 dist-test-slave-0t40 krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:50 dist-test-slave-0t40 krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 20:36:50.147194 958 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:50.141115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56626 (local address 127.25.62.190:42717)
0619 20:36:50.141173 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:50.141175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:50.141300 (+ 125us) server_negotiation.cc:407] Connection header received
0619 20:36:50.141415 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:50.141417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:50.141430 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:50.141505 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:50.142176 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.142691 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.143552 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.143748 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.143835 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:50.147107 (+ 3272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56626: BlockingRecv error: recv got EOF from 127.0.0.1:56626 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Jun 19 20:36:50 dist-test-slave-0t40 krb5kdc[864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM
I20260619 20:36:50.155673 958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:50.148661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56638 (local address 127.25.62.190:42717)
0619 20:36:50.148698 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:50.148700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:50.148708 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:50.148729 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:50.148730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:50.148748 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:50.148822 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:50.149346 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.149820 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.150517 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.150716 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.150804 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:50.152947 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:50.152949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:50.152949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:50.152957 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:50.154517 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:50.154903 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:50.154904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:50.154905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:50.154965 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:50.155169 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:50.155170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:50.155171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:50.155283 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:50.155404 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:50.155498 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 20:36:50.155572 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:36:50.157246 908 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:36:50.157424 908 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:36:50.158842 1168 tablet_service.cc:1467] Processing CreateTablet for tablet 85b502606882463dbc5c80a8b3946eb4 (DEFAULT_TABLE table=test-table [id=86d61279e321432d8785f445652aada4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:50.158866 1326 tablet_service.cc:1467] Processing CreateTablet for tablet 85b502606882463dbc5c80a8b3946eb4 (DEFAULT_TABLE table=test-table [id=86d61279e321432d8785f445652aada4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:50.158867 1046 tablet_service.cc:1467] Processing CreateTablet for tablet 85b502606882463dbc5c80a8b3946eb4 (DEFAULT_TABLE table=test-table [id=86d61279e321432d8785f445652aada4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:50.158977 1168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85b502606882463dbc5c80a8b3946eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:50.158996 1046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85b502606882463dbc5c80a8b3946eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:50.158998 1326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85b502606882463dbc5c80a8b3946eb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:50.160250 1421 tablet_bootstrap.cc:492] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1: Bootstrap starting.
I20260619 20:36:50.160250 1424 tablet_bootstrap.cc:492] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642: Bootstrap starting.
I20260619 20:36:50.160301 1420 tablet_bootstrap.cc:492] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3: Bootstrap starting.
I20260619 20:36:50.160554 1424 tablet_bootstrap.cc:654] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:50.160554 1421 tablet_bootstrap.cc:654] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:50.160728 1420 tablet_bootstrap.cc:654] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:50.161289 1421 tablet_bootstrap.cc:492] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1: No bootstrap required, opened a new log
I20260619 20:36:50.161346 1421 ts_tablet_manager.cc:1397] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:50.161377 1420 tablet_bootstrap.cc:492] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3: No bootstrap required, opened a new log
I20260619 20:36:50.161424 1420 ts_tablet_manager.cc:1397] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:50.161501 1421 raft_consensus.cc:348] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.161571 1421 raft_consensus.cc:374] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:50.161592 1421 raft_consensus.cc:729] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 954d742a8afe4e35b466850bfeef94d1, State: Initialized, Role: FOLLOWER
I20260619 20:36:50.161602 1424 tablet_bootstrap.cc:492] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642: No bootstrap required, opened a new log
I20260619 20:36:50.161582 1420 raft_consensus.cc:348] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.161643 1420 raft_consensus.cc:374] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:50.161645 1424 ts_tablet_manager.cc:1397] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:50.161635 1421 consensus_queue.cc:260] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.161664 1420 raft_consensus.cc:729] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8095c17dbb84f11acbdf8c1090822f3, State: Initialized, Role: FOLLOWER
I20260619 20:36:50.161706 1420 consensus_queue.cc:260] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.161729 1421 ts_tablet_manager.cc:1428] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:50.161747 1424 raft_consensus.cc:348] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.161810 1424 raft_consensus.cc:374] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:50.161816 1420 ts_tablet_manager.cc:1428] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:50.161834 1424 raft_consensus.cc:729] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31d910f6497f450dbecbd5cd3e6ab642, State: Initialized, Role: FOLLOWER
I20260619 20:36:50.161877 1424 consensus_queue.cc:260] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.161965 1424 ts_tablet_manager.cc:1428] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:50.208128 1471 raft_consensus.cc:482] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:36:50.208238 1471 raft_consensus.cc:504] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.208400 1471 leader_election.cc:288] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31d910f6497f450dbecbd5cd3e6ab642 (127.25.62.129:43649), f8095c17dbb84f11acbdf8c1090822f3 (127.25.62.131:44601)
I20260619 20:36:50.208587 1346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b502606882463dbc5c80a8b3946eb4" candidate_uuid: "954d742a8afe4e35b466850bfeef94d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8095c17dbb84f11acbdf8c1090822f3" is_pre_election: true
I20260619 20:36:50.208681 1346 raft_consensus.cc:2454] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 954d742a8afe4e35b466850bfeef94d1 in term 0.
I20260619 20:36:50.208591 1066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b502606882463dbc5c80a8b3946eb4" candidate_uuid: "954d742a8afe4e35b466850bfeef94d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31d910f6497f450dbecbd5cd3e6ab642" is_pre_election: true
I20260619 20:36:50.208726 1066 raft_consensus.cc:2454] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 954d742a8afe4e35b466850bfeef94d1 in term 0.
I20260619 20:36:50.208847 1140 leader_election.cc:302] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 954d742a8afe4e35b466850bfeef94d1, f8095c17dbb84f11acbdf8c1090822f3; no voters:
I20260619 20:36:50.208945 1471 raft_consensus.cc:2781] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:36:50.208971 1471 raft_consensus.cc:482] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:36:50.208997 1471 raft_consensus.cc:3037] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:50.209713 1471 raft_consensus.cc:504] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.209856 1471 leader_election.cc:288] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [CANDIDATE]: Term 1 election: Requested vote from peers 31d910f6497f450dbecbd5cd3e6ab642 (127.25.62.129:43649), f8095c17dbb84f11acbdf8c1090822f3 (127.25.62.131:44601)
I20260619 20:36:50.209981 1066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b502606882463dbc5c80a8b3946eb4" candidate_uuid: "954d742a8afe4e35b466850bfeef94d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31d910f6497f450dbecbd5cd3e6ab642"
I20260619 20:36:50.210052 1066 raft_consensus.cc:3037] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:50.210052 1346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b502606882463dbc5c80a8b3946eb4" candidate_uuid: "954d742a8afe4e35b466850bfeef94d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8095c17dbb84f11acbdf8c1090822f3"
I20260619 20:36:50.210134 1346 raft_consensus.cc:3037] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:50.210759 1066 raft_consensus.cc:2454] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 954d742a8afe4e35b466850bfeef94d1 in term 1.
I20260619 20:36:50.210929 1346 raft_consensus.cc:2454] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 954d742a8afe4e35b466850bfeef94d1 in term 1.
I20260619 20:36:50.210956 1138 leader_election.cc:302] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31d910f6497f450dbecbd5cd3e6ab642, 954d742a8afe4e35b466850bfeef94d1; no voters:
I20260619 20:36:50.211050 1471 raft_consensus.cc:2781] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:50.211099 1471 raft_consensus.cc:686] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [term 1 LEADER]: Becoming Leader. State: Replica: 954d742a8afe4e35b466850bfeef94d1, State: Running, Role: LEADER
I20260619 20:36:50.211153 1471 consensus_queue.cc:237] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.211766 907 catalog_manager.cc:5495] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 954d742a8afe4e35b466850bfeef94d1 (127.25.62.130). New cstate: current_term: 1 leader_uuid: "954d742a8afe4e35b466850bfeef94d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } health_report { overall_health: UNKNOWN } } }
I20260619 20:36:50.223394 1416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:50.221063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52702 (local address 127.25.62.130:46379)
0619 20:36:50.221108 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:50.221110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:50.221119 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:36:50.221173 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:50.221174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:50.221182 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:50.221225 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:50.221345 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.221797 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.222818 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.222997 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.223046 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:50.223194 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:50.223219 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:50.223251 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:50.223307 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 20:36:50.223320 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:36:50.224218 1066 raft_consensus.cc:1261] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 FOLLOWER]: Refusing update from remote peer 954d742a8afe4e35b466850bfeef94d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:50.224275 1346 raft_consensus.cc:1261] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3 [term 1 FOLLOWER]: Refusing update from remote peer 954d742a8afe4e35b466850bfeef94d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:50.224416 1426 consensus_queue.cc:1035] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:50.224632 1426 consensus_queue.cc:1035] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:50.227310 907 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:50.227394 907 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:50.228204 907 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653: Sending DeleteTablet for 3 replicas of tablet 85b502606882463dbc5c80a8b3946eb4
I20260619 20:36:50.228479 1172 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b502606882463dbc5c80a8b3946eb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:50 UTC) from {username='oryx', principal='oryx/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:52676
I20260619 20:36:50.228511 1168 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b502606882463dbc5c80a8b3946eb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:50 UTC) from {username='oryx', principal='oryx/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:52676
I20260619 20:36:50.228586 1046 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b502606882463dbc5c80a8b3946eb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:50 UTC) from {username='oryx', principal='oryx/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:47332
I20260619 20:36:50.228664 1445 tablet_replica.cc:330] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642: stopping tablet replica
I20260619 20:36:50.228664 1444 tablet_replica.cc:330] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1: stopping tablet replica
I20260619 20:36:50.228710 1445 raft_consensus.cc:2229] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:50.228710 1444 raft_consensus.cc:2229] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:36:50.228752 1445 raft_consensus.cc:2258] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:50.228796 1444 raft_consensus.cc:2258] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:50.228837 907 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:36:50.228974 907 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:36:50.229326 1326 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b502606882463dbc5c80a8b3946eb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:50 UTC) from {username='oryx', principal='oryx/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:54614
I20260619 20:36:50.229336 1325 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b502606882463dbc5c80a8b3946eb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:50 UTC) from {username='oryx', principal='oryx/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:54614
I20260619 20:36:50.229486 1443 tablet_replica.cc:330] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3: stopping tablet replica
I20260619 20:36:50.229529 1443 raft_consensus.cc:2229] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:50.229564 1443 raft_consensus.cc:2258] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:50.229571 1444 ts_tablet_manager.cc:1905] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:50.229573 1445 ts_tablet_manager.cc:1905] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:50.230350 1046 tablet_service.cc:1467] Processing CreateTablet for tablet 5d2d4b788d54418787486fb88bad3911 (DEFAULT_TABLE table=test-table [id=9774f7c9c2554a828d23eb444507192c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:50.230463 1046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d2d4b788d54418787486fb88bad3911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:50.231106 1445 ts_tablet_manager.cc:1918] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:36:50.231163 1445 log.cc:1196] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-0/wal/wals/85b502606882463dbc5c80a8b3946eb4
I20260619 20:36:50.231370 1445 ts_tablet_manager.cc:1939] T 85b502606882463dbc5c80a8b3946eb4 P 31d910f6497f450dbecbd5cd3e6ab642: Deleting consensus metadata
I20260619 20:36:50.231781 1424 tablet_bootstrap.cc:492] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642: Bootstrap starting.
I20260619 20:36:50.231820 1172 tablet_service.cc:1467] Processing CreateTablet for tablet 5d2d4b788d54418787486fb88bad3911 (DEFAULT_TABLE table=test-table [id=9774f7c9c2554a828d23eb444507192c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:50.231853 894 catalog_manager.cc:4841] TS 31d910f6497f450dbecbd5cd3e6ab642 (127.25.62.129:43649): tablet 85b502606882463dbc5c80a8b3946eb4 (table test-table [id=86d61279e321432d8785f445652aada4]) successfully deleted
I20260619 20:36:50.232084 1424 tablet_bootstrap.cc:654] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:50.232097 1444 ts_tablet_manager.cc:1918] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:36:50.232142 1444 log.cc:1196] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/85b502606882463dbc5c80a8b3946eb4
I20260619 20:36:50.232134 1325 tablet_service.cc:1467] Processing CreateTablet for tablet 5d2d4b788d54418787486fb88bad3911 (DEFAULT_TABLE table=test-table [id=9774f7c9c2554a828d23eb444507192c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:36:50.232239 1325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d2d4b788d54418787486fb88bad3911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:50.232370 1444 ts_tablet_manager.cc:1939] T 85b502606882463dbc5c80a8b3946eb4 P 954d742a8afe4e35b466850bfeef94d1: Deleting consensus metadata
I20260619 20:36:50.232503 1172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d2d4b788d54418787486fb88bad3911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:50.233546 1420 tablet_bootstrap.cc:492] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3: Bootstrap starting.
I20260619 20:36:50.233587 894 catalog_manager.cc:4841] TS 954d742a8afe4e35b466850bfeef94d1 (127.25.62.130:46379): tablet 85b502606882463dbc5c80a8b3946eb4 (table test-table [id=86d61279e321432d8785f445652aada4]) successfully deleted
I20260619 20:36:50.233610 1421 tablet_bootstrap.cc:492] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1: Bootstrap starting.
W20260619 20:36:50.233683 894 catalog_manager.cc:4808] TS 954d742a8afe4e35b466850bfeef94d1 (127.25.62.130:46379): delete failed for tablet 85b502606882463dbc5c80a8b3946eb4 because the tablet was not found. No further retry: Not found: Tablet not found: 85b502606882463dbc5c80a8b3946eb4
I20260619 20:36:50.233884 1421 tablet_bootstrap.cc:654] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:50.233929 1420 tablet_bootstrap.cc:654] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:50.234531 1421 tablet_bootstrap.cc:492] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1: No bootstrap required, opened a new log
I20260619 20:36:50.234581 1421 ts_tablet_manager.cc:1397] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:50.234717 1421 raft_consensus.cc:348] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.234795 1421 raft_consensus.cc:374] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:50.234822 1421 raft_consensus.cc:729] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 954d742a8afe4e35b466850bfeef94d1, State: Initialized, Role: FOLLOWER
I20260619 20:36:50.234870 1421 consensus_queue.cc:260] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.234967 1421 ts_tablet_manager.cc:1428] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:50.235181 1443 ts_tablet_manager.cc:1905] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:50.236265 1424 tablet_bootstrap.cc:492] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642: No bootstrap required, opened a new log
I20260619 20:36:50.236313 1424 ts_tablet_manager.cc:1397] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260619 20:36:50.236418 1424 raft_consensus.cc:348] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.236464 1424 raft_consensus.cc:374] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:50.236500 1424 raft_consensus.cc:729] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31d910f6497f450dbecbd5cd3e6ab642, State: Initialized, Role: FOLLOWER
I20260619 20:36:50.236547 1424 consensus_queue.cc:260] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.236637 1424 ts_tablet_manager.cc:1428] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:50.236927 1443 ts_tablet_manager.cc:1918] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:36:50.236996 1443 log.cc:1196] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/85b502606882463dbc5c80a8b3946eb4
I20260619 20:36:50.237227 1420 tablet_bootstrap.cc:492] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3: No bootstrap required, opened a new log
I20260619 20:36:50.237265 1420 ts_tablet_manager.cc:1397] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 20:36:50.237367 1420 raft_consensus.cc:348] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.237416 1420 raft_consensus.cc:374] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:50.237429 1420 raft_consensus.cc:729] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8095c17dbb84f11acbdf8c1090822f3, State: Initialized, Role: FOLLOWER
I20260619 20:36:50.237483 1420 consensus_queue.cc:260] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.237574 1420 ts_tablet_manager.cc:1428] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:50.237869 1443 ts_tablet_manager.cc:1939] T 85b502606882463dbc5c80a8b3946eb4 P f8095c17dbb84f11acbdf8c1090822f3: Deleting consensus metadata
I20260619 20:36:50.238210 896 catalog_manager.cc:4841] TS f8095c17dbb84f11acbdf8c1090822f3 (127.25.62.131:44601): tablet 85b502606882463dbc5c80a8b3946eb4 (table test-table [id=86d61279e321432d8785f445652aada4]) successfully deleted
W20260619 20:36:50.238278 896 catalog_manager.cc:4808] TS f8095c17dbb84f11acbdf8c1090822f3 (127.25.62.131:44601): delete failed for tablet 85b502606882463dbc5c80a8b3946eb4 because the tablet was not found. No further retry: Not found: Tablet not found: 85b502606882463dbc5c80a8b3946eb4
I20260619 20:36:50.364365 1428 raft_consensus.cc:482] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:36:50.364465 1428 raft_consensus.cc:504] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.364624 1428 leader_election.cc:288] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 954d742a8afe4e35b466850bfeef94d1 (127.25.62.130:46379), f8095c17dbb84f11acbdf8c1090822f3 (127.25.62.131:44601)
I20260619 20:36:50.364892 1346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d2d4b788d54418787486fb88bad3911" candidate_uuid: "31d910f6497f450dbecbd5cd3e6ab642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8095c17dbb84f11acbdf8c1090822f3" is_pre_election: true
I20260619 20:36:50.364904 1190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d2d4b788d54418787486fb88bad3911" candidate_uuid: "31d910f6497f450dbecbd5cd3e6ab642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "954d742a8afe4e35b466850bfeef94d1" is_pre_election: true
I20260619 20:36:50.364995 1346 raft_consensus.cc:2454] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31d910f6497f450dbecbd5cd3e6ab642 in term 0.
I20260619 20:36:50.364992 1190 raft_consensus.cc:2454] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31d910f6497f450dbecbd5cd3e6ab642 in term 0.
I20260619 20:36:50.365198 1000 leader_election.cc:302] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31d910f6497f450dbecbd5cd3e6ab642, f8095c17dbb84f11acbdf8c1090822f3; no voters:
I20260619 20:36:50.365319 1428 raft_consensus.cc:2781] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:36:50.365363 1428 raft_consensus.cc:482] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:36:50.365406 1428 raft_consensus.cc:3037] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:50.366046 1428 raft_consensus.cc:504] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.366179 1428 leader_election.cc:288] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [CANDIDATE]: Term 1 election: Requested vote from peers 954d742a8afe4e35b466850bfeef94d1 (127.25.62.130:46379), f8095c17dbb84f11acbdf8c1090822f3 (127.25.62.131:44601)
I20260619 20:36:50.366364 1346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d2d4b788d54418787486fb88bad3911" candidate_uuid: "31d910f6497f450dbecbd5cd3e6ab642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8095c17dbb84f11acbdf8c1090822f3"
I20260619 20:36:50.366384 1190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d2d4b788d54418787486fb88bad3911" candidate_uuid: "31d910f6497f450dbecbd5cd3e6ab642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "954d742a8afe4e35b466850bfeef94d1"
I20260619 20:36:50.366441 1346 raft_consensus.cc:3037] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:50.366463 1190 raft_consensus.cc:3037] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:36:50.367056 1190 raft_consensus.cc:2454] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31d910f6497f450dbecbd5cd3e6ab642 in term 1.
I20260619 20:36:50.367070 1346 raft_consensus.cc:2454] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31d910f6497f450dbecbd5cd3e6ab642 in term 1.
I20260619 20:36:50.367235 1000 leader_election.cc:302] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31d910f6497f450dbecbd5cd3e6ab642, f8095c17dbb84f11acbdf8c1090822f3; no voters:
I20260619 20:36:50.367327 1428 raft_consensus.cc:2781] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:50.367406 1428 raft_consensus.cc:686] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 LEADER]: Becoming Leader. State: Replica: 31d910f6497f450dbecbd5cd3e6ab642, State: Running, Role: LEADER
I20260619 20:36:50.367460 1428 consensus_queue.cc:237] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } }
I20260619 20:36:50.368031 907 catalog_manager.cc:5495] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31d910f6497f450dbecbd5cd3e6ab642 (127.25.62.129). New cstate: current_term: 1 leader_uuid: "31d910f6497f450dbecbd5cd3e6ab642" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31d910f6497f450dbecbd5cd3e6ab642" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 } health_report { overall_health: UNKNOWN } } }
I20260619 20:36:50.385006 1419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:50.382723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47338 (local address 127.25.62.129:43649)
0619 20:36:50.382779 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:50.382781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:50.382789 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:50.382812 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:50.382813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:50.382825 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:50.382869 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:36:50.383003 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.383559 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.384293 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:50.384554 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:50.384624 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:50.384761 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:36:50.384794 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:36:50.384841 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:50.384897 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 20:36:50.384918 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 20:36:50.387195 1346 raft_consensus.cc:1261] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3 [term 1 FOLLOWER]: Refusing update from remote peer 31d910f6497f450dbecbd5cd3e6ab642: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:50.387239 1190 raft_consensus.cc:1261] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1 [term 1 FOLLOWER]: Refusing update from remote peer 31d910f6497f450dbecbd5cd3e6ab642: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:36:50.387390 1428 consensus_queue.cc:1035] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8095c17dbb84f11acbdf8c1090822f3" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:50.387511 1440 consensus_queue.cc:1035] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "954d742a8afe4e35b466850bfeef94d1" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 46379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:36:50.399415 907 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:50.399529 907 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:36:50.400131 907 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 556c0f4af6ee4aeaaacd88eed02f0653: Sending DeleteTablet for 3 replicas of tablet 5d2d4b788d54418787486fb88bad3911
I20260619 20:36:50.400384 1325 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d2d4b788d54418787486fb88bad3911 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:50 UTC) from {username='oryx', principal='oryx/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:54614
I20260619 20:36:50.400403 1046 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d2d4b788d54418787486fb88bad3911 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:50 UTC) from {username='oryx', principal='oryx/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:47332
I20260619 20:36:50.400525 1172 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d2d4b788d54418787486fb88bad3911 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:36:50 UTC) from {username='oryx', principal='oryx/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:52676
I20260619 20:36:50.400555 1445 tablet_replica.cc:330] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642: stopping tablet replica
I20260619 20:36:50.400641 1445 raft_consensus.cc:2229] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:36:50.400642 1443 tablet_replica.cc:330] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3: stopping tablet replica
I20260619 20:36:50.400698 1443 raft_consensus.cc:2229] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:50.400699 1445 raft_consensus.cc:2258] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:50.400736 1443 raft_consensus.cc:2258] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:50.400868 1444 tablet_replica.cc:330] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1: stopping tablet replica
I20260619 20:36:50.400925 1444 raft_consensus.cc:2229] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:36:50.400929 1445 ts_tablet_manager.cc:1905] T 5d2d4b788d54418787486fb88bad3911 P 31d910f6497f450dbecbd5cd3e6ab642: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:50.400964 1444 raft_consensus.cc:2258] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:36:50.401299 1443 ts_tablet_manager.cc:1905] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:50.401356 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 981
I20260619 20:36:50.403007 1443 ts_tablet_manager.cc:1918] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:36:50.403060 1443 log.cc:1196] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/5d2d4b788d54418787486fb88bad3911
I20260619 20:36:50.403412 1443 ts_tablet_manager.cc:1939] T 5d2d4b788d54418787486fb88bad3911 P f8095c17dbb84f11acbdf8c1090822f3: Deleting consensus metadata
I20260619 20:36:50.403750 896 catalog_manager.cc:4841] TS f8095c17dbb84f11acbdf8c1090822f3 (127.25.62.131:44601): tablet 5d2d4b788d54418787486fb88bad3911 (table test-table [id=9774f7c9c2554a828d23eb444507192c]) successfully deleted
I20260619 20:36:50.403987 1444 ts_tablet_manager.cc:1905] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:36:50.405118 1444 ts_tablet_manager.cc:1918] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 20:36:50.405186 1444 log.cc:1196] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/5d2d4b788d54418787486fb88bad3911
I20260619 20:36:50.405400 1444 ts_tablet_manager.cc:1939] T 5d2d4b788d54418787486fb88bad3911 P 954d742a8afe4e35b466850bfeef94d1: Deleting consensus metadata
I20260619 20:36:50.405740 894 catalog_manager.cc:4841] TS 954d742a8afe4e35b466850bfeef94d1 (127.25.62.130:46379): tablet 5d2d4b788d54418787486fb88bad3911 (table test-table [id=9774f7c9c2554a828d23eb444507192c]) successfully deleted
W20260619 20:36:50.409564 894 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.62.129:43649 (error 108)
I20260619 20:36:50.409822 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 1121
I20260619 20:36:50.410127 1504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:50.409960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:43649 (local address 127.0.0.1:47348)
0619 20:36:50.410048 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:50.410084 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.129:43649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72,"thread_start_us":37,"threads_started":1}
W20260619 20:36:50.410203 894 catalog_manager.cc:4584] TS 31d910f6497f450dbecbd5cd3e6ab642 (127.25.62.129:43649): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5d2d4b788d54418787486fb88bad3911: Network error: Client connection negotiation failed: client connection to 127.25.62.129:43649: connect: Connection refused (error 111)
I20260619 20:36:50.416118 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 1261
I20260619 20:36:50.421391 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 877
2026-06-19T20:36:50Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2481 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:50 dist-test-slave-0t40 krb5kdc[1507](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:50 dist-test-slave-0t40 krb5kdc[1507](info): set up 2 sockets
Jun 19 20:36:50 dist-test-slave-0t40 krb5kdc[1507](info): commencing operation
krb5kdc: starting...
W20260619 20:36:51.465814 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:36:51 dist-test-slave-0t40 krb5kdc[1507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:36:51.582671 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:38161
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:38161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:51.661367 1520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:51.661572 1520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:51.661595 1520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:51.662750 1520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:51.662791 1520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:51.662806 1520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:51.662818 1520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:51.662830 1520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:51.664299 1520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:38161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:38161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:51.664546 1520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:51.664783 1520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:51.669535 1525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:51.669543 1526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:51.669612 1520 server_base.cc:1034] running on GCE node
W20260619 20:36:51.669538 1528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:51.670109 1520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:51.670338 1520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:51.671473 1520 hybrid_clock.cc:648] HybridClock initialized: now 1781901411671465 us; error 28 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T20:36:51Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1256 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:36:51 dist-test-slave-0t40 krb5kdc[1533](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:36:51 dist-test-slave-0t40 krb5kdc[1533](info): set up 2 sockets
Jun 19 20:36:51 dist-test-slave-0t40 krb5kdc[1533](info): commencing operation
krb5kdc: starting...
W20260619 20:36:52.721995 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:36:52 dist-test-slave-0t40 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:36:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:36:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:36:52.835830 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:37437
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:37437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:52.916760 1546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:52.916980 1546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:52.917004 1546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:52.918188 1546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:52.918236 1546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:52.918252 1546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:52.918267 1546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:52.918282 1546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:52.919682 1546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:37437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:37437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:52.919870 1546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:52.920042 1546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:52.924877 1554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:52.924860 1552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:52.924888 1551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:52.925134 1546 server_base.cc:1034] running on GCE node
I20260619 20:36:52.925444 1546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:52.925719 1546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:52.926891 1546 hybrid_clock.cc:648] HybridClock initialized: now 1781901412926868 us; error 35 us; skew 500 ppm
Jun 19 20:36:52 dist-test-slave-0t40 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901412, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:52.929364 1546 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:36:52.929858 1546 webserver.cc:460] Webserver started at http://127.25.62.190:40613/ using document root <none> and password file <none>
I20260619 20:36:52.930090 1546 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:52.930161 1546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:52.930321 1546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:52.931389 1546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "1589fdf887f7486aa3de46f0290a9649"
format_stamp: "Formatted at 2026-06-19 20:36:52 on dist-test-slave-0t40"
server_key: "6362fa257f70300bc57a8f0a1876366e"
server_key_iv: "87c2956839d3118608fc4c01e55ee9bc"
server_key_version: "encryptionkey@0"
I20260619 20:36:52.931767 1546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "1589fdf887f7486aa3de46f0290a9649"
format_stamp: "Formatted at 2026-06-19 20:36:52 on dist-test-slave-0t40"
server_key: "6362fa257f70300bc57a8f0a1876366e"
server_key_iv: "87c2956839d3118608fc4c01e55ee9bc"
server_key_version: "encryptionkey@0"
I20260619 20:36:52.933171 1546 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:36:52.933947 1561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:52.934139 1546 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:36:52.934254 1546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "1589fdf887f7486aa3de46f0290a9649"
format_stamp: "Formatted at 2026-06-19 20:36:52 on dist-test-slave-0t40"
server_key: "6362fa257f70300bc57a8f0a1876366e"
server_key_iv: "87c2956839d3118608fc4c01e55ee9bc"
server_key_version: "encryptionkey@0"
I20260619 20:36:52.934335 1546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:52.941581 1546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:52.943346 1546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:52.943518 1546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:52.947959 1546 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:37437
I20260619 20:36:52.948000 1613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:37437 every 8 connection(s)
I20260619 20:36:52.948382 1546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:52.949252 1614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:36:52.950204 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 1546
I20260619 20:36:52.950315 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:36:52.950496 25850 external_mini_cluster.cc:1428] Setting key 4948d00f555a1a21ef50a520325c1c44
I20260619 20:36:52.952637 1614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649: Bootstrap starting.
I20260619 20:36:52.953336 1614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649: Neither blocks nor log segments found. Creating new log.
I20260619 20:36:52.953632 1614 log.cc:824] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:52.954383 1614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649: No bootstrap required, opened a new log
I20260619 20:36:52.956944 1614 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1589fdf887f7486aa3de46f0290a9649" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37437 } }
I20260619 20:36:52.957166 1614 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:36:52.957194 1614 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1589fdf887f7486aa3de46f0290a9649, State: Initialized, Role: FOLLOWER
I20260619 20:36:52.957324 1614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1589fdf887f7486aa3de46f0290a9649" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37437 } }
I20260619 20:36:52.957401 1614 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:52.957466 1614 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:52.957515 1614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [term 0 FOLLOWER]: Advancing to term 1
Jun 19 20:36:52 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:52.958227 1614 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1589fdf887f7486aa3de46f0290a9649" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37437 } }
I20260619 20:36:52.958341 1614 leader_election.cc:302] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1589fdf887f7486aa3de46f0290a9649; no voters:
I20260619 20:36:52.958532 1614 leader_election.cc:288] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:36:52.958638 1619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:36:52.958801 1614 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:52.958868 1619 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [term 1 LEADER]: Becoming Leader. State: Replica: 1589fdf887f7486aa3de46f0290a9649, State: Running, Role: LEADER
I20260619 20:36:52.958997 1619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1589fdf887f7486aa3de46f0290a9649" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37437 } }
I20260619 20:36:52.959507 1620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1589fdf887f7486aa3de46f0290a9649" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1589fdf887f7486aa3de46f0290a9649" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37437 } } }
I20260619 20:36:52.959578 1621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1589fdf887f7486aa3de46f0290a9649. Latest consensus state: current_term: 1 leader_uuid: "1589fdf887f7486aa3de46f0290a9649" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1589fdf887f7486aa3de46f0290a9649" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37437 } } }
I20260619 20:36:52.959687 1621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:52.959657 1620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [sys.catalog]: This master's current role is: LEADER
W20260619 20:36:52.960657 1634 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:36:52.960739 1634 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:36:52.961485 1629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:52.961652 1629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:52.962587 1617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:52.951035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50806 (local address 127.25.62.190:37437)
0619 20:36:52.951224 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:52.951234 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:52.951277 (+ 43us) server_negotiation.cc:407] Connection header received
0619 20:36:52.951683 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:52.951689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:52.951740 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:52.951854 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:52.952655 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:52.953736 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:52.954766 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:52.955179 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:52.955264 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:52.958267 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:52.958271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:52.958271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:52.958285 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:52.960917 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:52.961375 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:52.961378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:52.961378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:52.961463 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:52.961697 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:52.961697 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:52.961698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:52.961943 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:52.962084 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:52.962209 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 20:36:52.962350 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":104,"threads_started":1}
I20260619 20:36:52.963948 1629 catalog_manager.cc:1347] Generated new cluster ID: 17285bb78572420abc28a59870cdaf65
I20260619 20:36:52.963995 1629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:52.975883 1629 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:36:52.976413 1629 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:52.979147 1629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649: Generated new TSK 0
I20260619 20:36:52.979310 1629 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
I20260619 20:36:53.007565 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:37437
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:53.089057 1640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:53.089257 1640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:53.089282 1640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:53.090373 1640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:53.090412 1640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:53.090440 1640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:36:53.091854 1640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:37437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:53.092044 1640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:53.092221 1640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:53.097196 1645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:53.097144 1648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:53.097134 1646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:53.097172 1640 server_base.cc:1034] running on GCE node
I20260619 20:36:53.097536 1640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:53.097791 1640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:53.098969 1640 hybrid_clock.cc:648] HybridClock initialized: now 1781901413098956 us; error 42 us; skew 500 ppm
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901413, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:53.101601 1640 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:36:53.102079 1640 webserver.cc:460] Webserver started at http://127.25.62.129:44741/ using document root <none> and password file <none>
I20260619 20:36:53.102298 1640 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:53.102365 1640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:53.102504 1640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:53.103489 1640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "0fe5019854574cb9941382cfea7de770"
format_stamp: "Formatted at 2026-06-19 20:36:53 on dist-test-slave-0t40"
server_key: "a9c489a83cb405b79e253745a830f9cf"
server_key_iv: "6f692d267b4fdf45fc4c2534f29ba0bb"
server_key_version: "encryptionkey@0"
I20260619 20:36:53.103847 1640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "0fe5019854574cb9941382cfea7de770"
format_stamp: "Formatted at 2026-06-19 20:36:53 on dist-test-slave-0t40"
server_key: "a9c489a83cb405b79e253745a830f9cf"
server_key_iv: "6f692d267b4fdf45fc4c2534f29ba0bb"
server_key_version: "encryptionkey@0"
I20260619 20:36:53.105177 1640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:36:53.105893 1655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:53.106076 1640 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:53.106170 1640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "0fe5019854574cb9941382cfea7de770"
format_stamp: "Formatted at 2026-06-19 20:36:53 on dist-test-slave-0t40"
server_key: "a9c489a83cb405b79e253745a830f9cf"
server_key_iv: "6f692d267b4fdf45fc4c2534f29ba0bb"
server_key_version: "encryptionkey@0"
I20260619 20:36:53.106257 1640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:53.115566 1640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:53.117854 1640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:53.118021 1640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:53.118240 1640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:53.118548 1640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:53.118604 1640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:53.118643 1640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:53.118670 1640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:53.124785 1640 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:34183
I20260619 20:36:53.124836 1768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:34183 every 8 connection(s)
I20260619 20:36:53.125142 1640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901413, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:53.132421 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 1640
I20260619 20:36:53.132553 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:36:53.132741 25850 external_mini_cluster.cc:1428] Setting key 83eea382169e2f9db40f1d6f821ad3e5
I20260619 20:36:53.133867 1617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:53.125815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:34465 (local address 127.25.62.190:37437)
0619 20:36:53.125878 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:53.125881 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:53.126058 (+ 177us) server_negotiation.cc:407] Connection header received
0619 20:36:53.126512 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:53.126512 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:53.126522 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:53.126583 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:53.127443 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.127920 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.128831 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.129029 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.129093 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:53.131120 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:53.131121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:53.131122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:53.131128 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:53.132355 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:53.132839 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:53.132841 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:53.132841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:53.132902 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:53.133195 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:53.133196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:53.133197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:53.133327 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:53.133420 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:53.133728 (+ 308us) server_negotiation.cc:299] Negotiation successful
0619 20:36:53.133803 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 20:36:53.134136 1771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:53.125789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.129:34465)
0619 20:36:53.125932 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:53.125971 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:53.126434 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:53.126633 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:53.126636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:53.127164 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:53.127352 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:53.127356 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.127996 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.127997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:53.128761 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:53.128763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.129120 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.129121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:53.129219 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:53.129713 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:53.129721 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:53.131032 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:53.132437 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:53.132443 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:53.132444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:53.132744 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:53.132975 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:53.132976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:53.132976 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:53.133127 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:53.133411 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:53.133417 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:53.133571 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 20:36:53.133691 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 20:36:53.133856 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
I20260619 20:36:53.134675 1769 heartbeater.cc:347] Connected to a master server at 127.25.62.190:37437
I20260619 20:36:53.134765 1769 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:53.134964 1769 heartbeater.cc:511] Master 127.25.62.190:37437 requested a full tablet report, sending...
I20260619 20:36:53.135470 1578 ts_manager.cc:194] Registered new tserver with Master: 0fe5019854574cb9941382cfea7de770 (127.25.62.129:34183)
I20260619 20:36:53.136053 1578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:34465
WARNING: no policy specified for kudu/127.25.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
I20260619 20:36:53.157411 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:37437
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:53.236109 1774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:53.236322 1774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:53.236356 1774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:53.237619 1774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:53.237679 1774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:53.237721 1774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:36:53.239125 1774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:37437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:53.239380 1774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:53.239602 1774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:53.244277 1779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:53.244393 1780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:53.244303 1782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:53.244518 1774 server_base.cc:1034] running on GCE node
I20260619 20:36:53.244745 1774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:53.245011 1774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:53.246201 1774 hybrid_clock.cc:648] HybridClock initialized: now 1781901413246195 us; error 34 us; skew 500 ppm
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901413, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:53.248479 1774 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:36:53.248891 1774 webserver.cc:460] Webserver started at http://127.25.62.130:34811/ using document root <none> and password file <none>
I20260619 20:36:53.249202 1774 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:53.249270 1774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:53.249401 1774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:53.250384 1774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "5aba7cea18cb4a36b4cb3a6afd680a55"
format_stamp: "Formatted at 2026-06-19 20:36:53 on dist-test-slave-0t40"
server_key: "c2e849a262d0d0bd5f2f314299153179"
server_key_iv: "acc1969a979199c5f716a502ad663fa7"
server_key_version: "encryptionkey@0"
I20260619 20:36:53.250744 1774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "5aba7cea18cb4a36b4cb3a6afd680a55"
format_stamp: "Formatted at 2026-06-19 20:36:53 on dist-test-slave-0t40"
server_key: "c2e849a262d0d0bd5f2f314299153179"
server_key_iv: "acc1969a979199c5f716a502ad663fa7"
server_key_version: "encryptionkey@0"
I20260619 20:36:53.252130 1774 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:36:53.252794 1789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:53.252966 1774 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:36:53.253077 1774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "5aba7cea18cb4a36b4cb3a6afd680a55"
format_stamp: "Formatted at 2026-06-19 20:36:53 on dist-test-slave-0t40"
server_key: "c2e849a262d0d0bd5f2f314299153179"
server_key_iv: "acc1969a979199c5f716a502ad663fa7"
server_key_version: "encryptionkey@0"
I20260619 20:36:53.253165 1774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:53.273250 1774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:53.275223 1774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:53.275393 1774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:53.275632 1774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:53.275969 1774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:53.276024 1774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:53.276064 1774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:53.276091 1774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:53.282099 1774 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:34061
I20260619 20:36:53.282157 1902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:34061 every 8 connection(s)
I20260619 20:36:53.282501 1774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901413, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:53.291579 1617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:53.283258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:55065 (local address 127.25.62.190:37437)
0619 20:36:53.283340 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:53.283342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:53.283503 (+ 161us) server_negotiation.cc:407] Connection header received
0619 20:36:53.284038 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:53.284040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:53.284054 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:53.284095 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:53.285135 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.285654 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.286655 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.286869 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.286950 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:53.289027 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:53.289030 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:53.289030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:53.289056 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:53.290229 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:53.290635 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:53.290637 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:53.290637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:53.290697 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:53.290950 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:53.290951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:53.290952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:53.291067 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:53.291144 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:53.291400 (+ 256us) server_negotiation.cc:299] Negotiation successful
0619 20:36:53.291490 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 20:36:53.291744 1905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:53.283156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.130:55065)
0619 20:36:53.283370 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:53.283422 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:53.283925 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:53.284178 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:53.284186 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:53.284787 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:53.284990 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:53.284995 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.285751 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.285752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:53.286551 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:53.286553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.286955 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.286956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:53.287081 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:53.287606 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:53.287614 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:53.288932 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:53.290324 (+ 1392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:53.290331 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:53.290333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:53.290548 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:53.290763 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:53.290764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:53.290764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:53.290865 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:53.291134 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:53.291140 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:53.291274 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 20:36:53.291379 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 20:36:53.291526 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":108,"threads_started":1}
I20260619 20:36:53.292119 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 1774
I20260619 20:36:53.292228 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:36:53.292351 1903 heartbeater.cc:347] Connected to a master server at 127.25.62.190:37437
I20260619 20:36:53.292433 1903 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:53.292447 25850 external_mini_cluster.cc:1428] Setting key e8c2638848fafa9775051b68b33f1b53
I20260619 20:36:53.292641 1903 heartbeater.cc:511] Master 127.25.62.190:37437 requested a full tablet report, sending...
I20260619 20:36:53.293020 1578 ts_manager.cc:194] Registered new tserver with Master: 5aba7cea18cb4a36b4cb3a6afd680a55 (127.25.62.130:34061)
I20260619 20:36:53.293545 1578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:55065
WARNING: no policy specified for kudu/127.25.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
I20260619 20:36:53.316115 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:37437
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:53.394402 1908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:53.394582 1908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:53.394618 1908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:53.395668 1908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:53.395699 1908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:53.395725 1908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:36:53.397608 1908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:37437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:53.397781 1908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:53.397940 1908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:36:53.402719 1908 server_base.cc:1034] running on GCE node
W20260619 20:36:53.402709 1916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:53.402693 1913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:53.402704 1914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:53.403121 1908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:53.403359 1908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:53.404551 1908 hybrid_clock.cc:648] HybridClock initialized: now 1781901413404544 us; error 34 us; skew 500 ppm
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901413, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:53.407188 1908 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:36:53.407603 1908 webserver.cc:460] Webserver started at http://127.25.62.131:36075/ using document root <none> and password file <none>
I20260619 20:36:53.407829 1908 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:53.407905 1908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:53.408079 1908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:36:53.409173 1908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "d3582a12eb374cd6bec3c96af4c9570d"
format_stamp: "Formatted at 2026-06-19 20:36:53 on dist-test-slave-0t40"
server_key: "81ac2a65c7631fe4ab375223ec203726"
server_key_iv: "b615d911add15cc94e141c9807c21473"
server_key_version: "encryptionkey@0"
I20260619 20:36:53.409550 1908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "d3582a12eb374cd6bec3c96af4c9570d"
format_stamp: "Formatted at 2026-06-19 20:36:53 on dist-test-slave-0t40"
server_key: "81ac2a65c7631fe4ab375223ec203726"
server_key_iv: "b615d911add15cc94e141c9807c21473"
server_key_version: "encryptionkey@0"
I20260619 20:36:53.410887 1908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:36:53.411640 1923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:53.411850 1908 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:53.411964 1908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "d3582a12eb374cd6bec3c96af4c9570d"
format_stamp: "Formatted at 2026-06-19 20:36:53 on dist-test-slave-0t40"
server_key: "81ac2a65c7631fe4ab375223ec203726"
server_key_iv: "b615d911add15cc94e141c9807c21473"
server_key_version: "encryptionkey@0"
I20260619 20:36:53.412052 1908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:53.427199 1908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:53.429546 1908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:53.429715 1908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:53.429958 1908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:53.430267 1908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:53.430322 1908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:53.430365 1908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:53.430393 1908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:53.437107 1908 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:34047
I20260619 20:36:53.437160 2036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:34047 every 8 connection(s)
I20260619 20:36:53.437531 1908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:36:53.440590 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 1908
I20260619 20:36:53.440701 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:36:53.440882 25850 external_mini_cluster.cc:1428] Setting key ab86004fed4935ce811d7809c60a1d0c
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901413, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:36:53.446609 1617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:53.438237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:40783 (local address 127.25.62.190:37437)
0619 20:36:53.438315 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:53.438318 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:53.438514 (+ 196us) server_negotiation.cc:407] Connection header received
0619 20:36:53.438977 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:53.438978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:53.438988 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:53.439040 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:53.439896 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.440388 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.441724 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.441938 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.442025 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:53.444091 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:53.444092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:53.444092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:53.444100 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:53.445539 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:53.445896 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:53.445896 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:53.445897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:53.445949 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:53.446147 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:53.446148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:53.446148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:53.446243 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:53.446291 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:53.446493 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 20:36:53.446549 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 20:36:53.446852 2039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:53.438211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.131:40783)
0619 20:36:53.438399 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:53.438433 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:53.438897 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:53.439092 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:53.439095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:53.439626 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:53.439811 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:53.439815 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.440488 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.440488 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:53.441618 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:53.441620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.442034 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.442035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:53.442171 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:53.442686 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:53.442694 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:53.443993 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:36:53.445611 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:53.445617 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:53.445618 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:53.445817 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:53.446005 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:36:53.446005 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:36:53.446006 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:36:53.446096 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:36:53.446294 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:36:53.446298 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:36:53.446404 (+ 106us) client_negotiation.cc:769] Sending connection context
0619 20:36:53.446483 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 20:36:53.446593 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":106,"threads_started":1}
I20260619 20:36:53.447454 2037 heartbeater.cc:347] Connected to a master server at 127.25.62.190:37437
I20260619 20:36:53.447559 2037 heartbeater.cc:464] Registering TS with master...
I20260619 20:36:53.447731 2037 heartbeater.cc:511] Master 127.25.62.190:37437 requested a full tablet report, sending...
I20260619 20:36:53.448096 1578 ts_manager.cc:194] Registered new tserver with Master: d3582a12eb374cd6bec3c96af4c9570d (127.25.62.131:34047)
I20260619 20:36:53.448478 1578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:40783
I20260619 20:36:53.452616 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.129.keytab.
Entry for principal oryx/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.129.keytab.
WARNING: no policy specified for oryx/127.25.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.130.keytab.
Entry for principal oryx/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.130.keytab.
WARNING: no policy specified for oryx/127.25.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.25.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.131.keytab.
Entry for principal oryx/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.131.keytab.
I20260619 20:36:53.523303 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 1640
I20260619 20:36:53.527689 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 1774
I20260619 20:36:53.531800 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 1908
I20260619 20:36:53.535873 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 1546
I20260619 20:36:53.541131 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:37437
--webserver_interface=127.25.62.190
--webserver_port=40613
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:37437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:36:53.620548 2046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:53.620781 2046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:53.620816 2046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:53.621973 2046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:36:53.622042 2046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:53.622067 2046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:53.622109 2046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:36:53.622133 2046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:36:53.623549 2046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:37437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:37437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=40613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:53.623813 2046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:53.624029 2046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:53.628741 2054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:53.628739 2052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:53.628705 2051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:53.628767 2046 server_base.cc:1034] running on GCE node
I20260619 20:36:53.629176 2046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:53.629458 2046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:53.630625 2046 hybrid_clock.cc:648] HybridClock initialized: now 1781901413630620 us; error 35 us; skew 500 ppm
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901413, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:53.632897 2046 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:36:53.633401 2046 webserver.cc:460] Webserver started at http://127.25.62.190:40613/ using document root <none> and password file <none>
I20260619 20:36:53.633620 2046 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:53.633700 2046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:53.634996 2046 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:36:53.635648 2061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:53.635808 2046 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:36:53.635903 2046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "1589fdf887f7486aa3de46f0290a9649"
format_stamp: "Formatted at 2026-06-19 20:36:52 on dist-test-slave-0t40"
server_key: "6362fa257f70300bc57a8f0a1876366e"
server_key_iv: "87c2956839d3118608fc4c01e55ee9bc"
server_key_version: "encryptionkey@0"
I20260619 20:36:53.636242 2046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:53.661686 2046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:53.663595 2046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:53.663779 2046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:53.668025 2046 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:37437
I20260619 20:36:53.668054 2113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:37437 every 8 connection(s)
I20260619 20:36:53.668440 2046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:36:53.669911 2114 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:36:53.670481 2114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649: Bootstrap starting.
I20260619 20:36:53.675645 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 2046
I20260619 20:36:53.676000 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:34183
--local_ip_for_outbound_sockets=127.25.62.129
--tserver_master_addrs=127.25.62.190:37437
--webserver_port=44741
--webserver_interface=127.25.62.129
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.62.129
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:36:53.679145 2114 log.cc:824] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649: Log is configured to *not* fsync() on all Append() calls
I20260619 20:36:53.680814 2114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:36:53.681162 2114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649: Bootstrap complete.
I20260619 20:36:53.683367 2114 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1589fdf887f7486aa3de46f0290a9649" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37437 } }
I20260619 20:36:53.683578 2114 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1589fdf887f7486aa3de46f0290a9649, State: Initialized, Role: FOLLOWER
I20260619 20:36:53.683741 2114 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1589fdf887f7486aa3de46f0290a9649" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37437 } }
I20260619 20:36:53.683808 2114 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:36:53.683867 2114 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:36:53.683907 2114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [term 1 FOLLOWER]: Advancing to term 2
I20260619 20:36:53.684855 2114 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1589fdf887f7486aa3de46f0290a9649" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37437 } }
I20260619 20:36:53.684970 2114 leader_election.cc:302] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1589fdf887f7486aa3de46f0290a9649; no voters:
I20260619 20:36:53.685133 2114 leader_election.cc:288] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 20:36:53.685205 2118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 20:36:53.685357 2114 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:36:53.685448 2118 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [term 2 LEADER]: Becoming Leader. State: Replica: 1589fdf887f7486aa3de46f0290a9649, State: Running, Role: LEADER
I20260619 20:36:53.685567 2118 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1589fdf887f7486aa3de46f0290a9649" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37437 } }
I20260619 20:36:53.685832 2119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1589fdf887f7486aa3de46f0290a9649" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1589fdf887f7486aa3de46f0290a9649" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37437 } } }
I20260619 20:36:53.685940 2119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:53.685832 2120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1589fdf887f7486aa3de46f0290a9649. Latest consensus state: current_term: 2 leader_uuid: "1589fdf887f7486aa3de46f0290a9649" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1589fdf887f7486aa3de46f0290a9649" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37437 } } }
I20260619 20:36:53.686244 2120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649 [sys.catalog]: This master's current role is: LEADER
I20260619 20:36:53.686352 2127 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:36:53.686817 2127 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:36:53.686932 2127 catalog_manager.cc:1259] Loaded cluster ID: 17285bb78572420abc28a59870cdaf65
I20260619 20:36:53.686964 2127 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:36:53.688334 2127 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:36:53.688506 2127 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1589fdf887f7486aa3de46f0290a9649: Loaded TSK: 0
I20260619 20:36:53.688707 2127 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 20:36:53.757422 2116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:53.757607 2116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:53.757630 2116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:53.758765 2116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:53.758805 2116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:53.758831 2116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:36:53.760205 2116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.129.keytab
--principal=oryx/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:34183
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=44741
--tserver_master_addrs=127.25.62.190:37437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:53.760429 2116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:53.760664 2116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:53.765517 2143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:53.765530 2140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:53.765533 2141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:53.766026 2116 server_base.cc:1034] running on GCE node
I20260619 20:36:53.766306 2116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:53.766542 2116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:53.767707 2116 hybrid_clock.cc:648] HybridClock initialized: now 1781901413767689 us; error 34 us; skew 500 ppm
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901413, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:53.770313 2116 init.cc:377] Logged in from keytab as oryx/127.25.62.129@KRBTEST.COM (short username oryx)
I20260619 20:36:53.770783 2116 webserver.cc:460] Webserver started at http://127.25.62.129:44741/ using document root <none> and password file <none>
I20260619 20:36:53.771037 2116 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:53.771095 2116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:53.772418 2116 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:53.773028 2150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:53.773241 2116 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:53.773325 2116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "0fe5019854574cb9941382cfea7de770"
format_stamp: "Formatted at 2026-06-19 20:36:53 on dist-test-slave-0t40"
server_key: "a9c489a83cb405b79e253745a830f9cf"
server_key_iv: "6f692d267b4fdf45fc4c2534f29ba0bb"
server_key_version: "encryptionkey@0"
I20260619 20:36:53.773653 2116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:53.797742 2116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:53.800017 2116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:53.800168 2116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:53.800446 2116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:53.800827 2116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:53.800894 2116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:53.800933 2116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:53.800964 2116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:53.808084 2116 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:34183
I20260619 20:36:53.808141 2263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:34183 every 8 connection(s)
I20260619 20:36:53.808508 2116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:36:53.811175 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 2116
I20260619 20:36:53.811470 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:34061
--local_ip_for_outbound_sockets=127.25.62.130
--tserver_master_addrs=127.25.62.190:37437
--webserver_port=34811
--webserver_interface=127.25.62.130
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.62.130
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:53.817277 2266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:53.809279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.129:47159)
0619 20:36:53.809463 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:53.809511 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:53.810058 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:53.810501 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:53.810505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:53.811099 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:53.811352 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:53.811363 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.812624 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.812626 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:53.813513 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:53.813516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.814348 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.814349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:53.814448 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:53.814968 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:53.814976 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:53.817133 (+ 2157us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
W20260619 20:36:53.817515 2267 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:53.809303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:47159 (local address 127.25.62.190:37437)
0619 20:36:53.809493 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:53.809503 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:53.809596 (+ 93us) server_negotiation.cc:407] Connection header received
0619 20:36:53.810262 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:53.810268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:53.810318 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:53.810464 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:53.811471 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.812505 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.813595 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.814258 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.814371 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:53.817308 (+ 2937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.129:47159: BlockingRecv error: recv got EOF from 127.25.62.129:47159 (error 108)
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
W20260619 20:36:53.817659 2264 heartbeater.cc:644] Failed to heartbeat to 127.25.62.190:37437 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.62.190:37437: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:53.821977 2266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:53.817887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.129:49475)
0619 20:36:53.817940 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:53.817948 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:53.818001 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:53.818119 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:53.818120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:53.818243 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:53.818282 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:53.818283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.818905 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.818906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:53.819396 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:53.819398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.819759 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.819760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:53.819823 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:53.820232 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:53.820237 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:53.821912 (+ 1675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 20:36:53.822069 2267 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:53.817942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:49475 (local address 127.25.62.190:37437)
0619 20:36:53.818001 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:53.818003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:53.818012 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:36:53.818057 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:53.818058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:53.818069 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:53.818119 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:53.818356 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.818814 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.819491 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.819678 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.819759 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:53.821993 (+ 2234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.129:49475: BlockingRecv error: recv got EOF from 127.25.62.129:49475 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:53.826517 2266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:53.822340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.129:41899)
0619 20:36:53.822444 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:53.822455 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:53.822505 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:53.822612 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:53.822613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:53.822745 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:53.822774 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:53.822775 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.823318 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.823319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:53.824120 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:53.824122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.824425 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.824426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:53.824479 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:53.824838 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:53.824842 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:53.826454 (+ 1612us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90}
W20260619 20:36:53.826588 2267 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:53.822403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:41899 (local address 127.25.62.190:37437)
0619 20:36:53.822443 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:53.822445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:53.822491 (+ 46us) server_negotiation.cc:407] Connection header received
0619 20:36:53.822558 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:53.822558 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:53.822566 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:53.822611 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:53.822833 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.823252 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.824196 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.824359 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.824407 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:53.826525 (+ 2118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.129:41899: BlockingRecv error: recv got EOF from 127.25.62.129:41899 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 20:36:53.826757 2264 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 20:36:53.893151 2268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:53.893347 2268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:53.893371 2268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:53.894524 2268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:53.894570 2268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:53.894618 2268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:36:53.896124 2268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.130.keytab
--principal=oryx/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:34061
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=34811
--tserver_master_addrs=127.25.62.190:37437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:53.896353 2268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:53.896699 2268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:53.901798 2276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:53.901753 2274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:53.902088 2268 server_base.cc:1034] running on GCE node
W20260619 20:36:53.901753 2273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:53.902346 2268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:53.902594 2268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:53.903751 2268 hybrid_clock.cc:648] HybridClock initialized: now 1781901413903742 us; error 47 us; skew 500 ppm
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901413, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:53.906281 2268 init.cc:377] Logged in from keytab as oryx/127.25.62.130@KRBTEST.COM (short username oryx)
I20260619 20:36:53.906733 2268 webserver.cc:460] Webserver started at http://127.25.62.130:34811/ using document root <none> and password file <none>
I20260619 20:36:53.906968 2268 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:53.907048 2268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:53.908325 2268 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:53.908922 2283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:53.909129 2268 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:53.909221 2268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "5aba7cea18cb4a36b4cb3a6afd680a55"
format_stamp: "Formatted at 2026-06-19 20:36:53 on dist-test-slave-0t40"
server_key: "c2e849a262d0d0bd5f2f314299153179"
server_key_iv: "acc1969a979199c5f716a502ad663fa7"
server_key_version: "encryptionkey@0"
I20260619 20:36:53.909550 2268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:53.936960 2268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:53.938898 2268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:53.939065 2268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:53.939287 2268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:53.939599 2268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:53.939652 2268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:53.939689 2268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:53.939716 2268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:53.945227 2268 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:34061
I20260619 20:36:53.945307 2396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:34061 every 8 connection(s)
I20260619 20:36:53.945627 2268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:36:53.946646 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 2268
I20260619 20:36:53.946957 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:34047
--local_ip_for_outbound_sockets=127.25.62.131
--tserver_master_addrs=127.25.62.190:37437
--webserver_port=36075
--webserver_interface=127.25.62.131
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.25.62.131
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 20:36:53.954145 2267 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:53.946360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:53821 (local address 127.25.62.190:37437)
0619 20:36:53.946482 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:53.946484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:53.946619 (+ 135us) server_negotiation.cc:407] Connection header received
0619 20:36:53.947247 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:53.947248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:53.947267 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:53.947332 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:53.948229 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.948730 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.950274 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.950562 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.950626 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:53.954098 (+ 3472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.130:53821: BlockingRecv error: recv got EOF from 127.25.62.130:53821 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260619 20:36:53.954245 2399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:53.946336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.130:53821)
0619 20:36:53.946489 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:53.946528 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:53.947122 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:53.947399 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:53.947402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:53.947946 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:53.948125 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:53.948130 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.948832 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.948833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:53.950182 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:53.950185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.950656 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.950657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:53.950813 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:53.951306 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:53.951316 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:53.954017 (+ 2701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
W20260619 20:36:53.954442 2397 heartbeater.cc:644] Failed to heartbeat to 127.25.62.190:37437 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.62.190:37437: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:53.959156 2399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:53.954657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.130:55663)
0619 20:36:53.954708 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:53.954719 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:53.954758 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:53.954869 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:53.954869 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:53.955029 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:53.955083 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:53.955085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.955668 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.955668 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:53.956569 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:53.956572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.956893 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.956894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:53.956969 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:53.957335 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:53.957341 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:53.959093 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 20:36:53.959250 2267 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:53.954714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:55663 (local address 127.25.62.190:37437)
0619 20:36:53.954750 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:53.954752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:53.954760 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:53.954811 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:53.954812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:53.954818 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:53.954864 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:53.955154 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.955591 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.956649 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.956816 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.956863 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:53.959169 (+ 2306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.130:55663: BlockingRecv error: recv got EOF from 127.25.62.130:55663 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:53 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:53.963660 2399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:53.959537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.130:49331)
0619 20:36:53.959585 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:53.959595 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:53.959657 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:53.959793 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:53.959793 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:53.959915 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:53.959939 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:53.959940 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.960580 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.960581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:53.961122 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:53.961124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.961496 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.961497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:53.961548 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:53.961962 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:53.961969 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:53.963585 (+ 1616us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 20:36:53.963709 2267 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:53.959604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:49331 (local address 127.25.62.190:37437)
0619 20:36:53.959658 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:53.959660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:53.959669 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:36:53.959710 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:53.959711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:53.959720 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:53.959790 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:53.960024 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.960506 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.961212 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:53.961392 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:53.961497 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:53.963650 (+ 2153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.130:49331: BlockingRecv error: recv got EOF from 127.25.62.130:49331 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260619 20:36:53.963773 2397 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 20:36:54.031052 2400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:36:54.031298 2400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:36:54.031338 2400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:36:54.032475 2400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:36:54.032552 2400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:36:54.032599 2400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:36:54.034089 2400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/krb5kdc/oryx_127.25.62.131.keytab
--principal=oryx/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:34047
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=36075
--tserver_master_addrs=127.25.62.190:37437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:36:54.034363 2400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:36:54.034622 2400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:36:54.039683 2408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:54.039677 2405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:36:54.039678 2406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:36:54.040071 2400 server_base.cc:1034] running on GCE node
I20260619 20:36:54.040267 2400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:36:54.040503 2400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:36:54.041642 2400 hybrid_clock.cc:648] HybridClock initialized: now 1781901414041639 us; error 37 us; skew 500 ppm
Jun 19 20:36:54 dist-test-slave-0t40 krb5kdc[1533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901414, etypes {rep=17 tkt=17 ses=17}, oryx/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:36:54.043967 2400 init.cc:377] Logged in from keytab as oryx/127.25.62.131@KRBTEST.COM (short username oryx)
I20260619 20:36:54.044369 2400 webserver.cc:460] Webserver started at http://127.25.62.131:36075/ using document root <none> and password file <none>
I20260619 20:36:54.044603 2400 fs_manager.cc:359] Metadata directory not provided
I20260619 20:36:54.044683 2400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:36:54.046030 2400 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:54.046680 2415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:54.046866 2400 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:36:54.046969 2400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "d3582a12eb374cd6bec3c96af4c9570d"
format_stamp: "Formatted at 2026-06-19 20:36:53 on dist-test-slave-0t40"
server_key: "81ac2a65c7631fe4ab375223ec203726"
server_key_iv: "b615d911add15cc94e141c9807c21473"
server_key_version: "encryptionkey@0"
I20260619 20:36:54.047312 2400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:36:54.062675 2400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:36:54.064477 2400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:36:54.064651 2400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:36:54.064870 2400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:36:54.065235 2400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:36:54.065287 2400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:54.065321 2400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:36:54.065348 2400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:36:54.071231 2400 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:34047
I20260619 20:36:54.071282 2528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:34047 every 8 connection(s)
I20260619 20:36:54.071652 2400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:36:54.072166 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 2400
Jun 19 20:36:54 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:54 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 20:36:54.079155 2267 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:54.072454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:60795 (local address 127.25.62.190:37437)
0619 20:36:54.072506 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:54.072509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:54.072711 (+ 202us) server_negotiation.cc:407] Connection header received
0619 20:36:54.073314 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:54.073315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:54.073326 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:54.073394 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:54.074247 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.074737 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.075942 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.076155 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.076239 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:54.079100 (+ 2861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.131:60795: BlockingRecv error: recv got EOF from 127.25.62.131:60795 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:36:54.079214 2531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:54.072411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.131:60795)
0619 20:36:54.072594 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:54.072632 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:54.073217 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:54.073445 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:54.073448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:54.073976 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:54.074162 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:54.074166 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.074826 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.074827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:54.075853 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:54.075856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.076250 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.076251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:54.076373 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:54.076823 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:54.076831 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:54.079029 (+ 2198us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
W20260619 20:36:54.079401 2529 heartbeater.cc:644] Failed to heartbeat to 127.25.62.190:37437 (0 consecutive failures): Not authorized: Failed to ping master at 127.25.62.190:37437: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
I20260619 20:36:54.080539 2533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:54.072716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50822 (local address 127.25.62.190:37437)
0619 20:36:54.072822 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:54.072824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:54.072830 (+ 6us) server_negotiation.cc:407] Connection header received
0619 20:36:54.072899 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:54.072900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:54.072912 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:54.072992 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:54.073723 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.074268 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.075182 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.075369 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.075497 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:54.077394 (+ 1897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:36:54.077399 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:36:54.077399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:36:54.077409 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:36:54.078998 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:54.079459 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:54.079462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:54.079463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:54.079590 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:36:54.079817 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:36:54.079818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:36:54.079819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:36:54.080054 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:36:54.080200 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 20:36:54.080316 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 20:36:54.080434 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":60,"threads_started":1}
Jun 19 20:36:54 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:54 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:54.083793 2531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:54.079629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.131:41389)
0619 20:36:54.079669 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:54.079678 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:54.079733 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:54.079853 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:54.079854 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:54.079972 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:54.080008 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:54.080010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.080704 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.080705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:54.081473 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:54.081475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.081763 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.081763 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:54.081809 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:54.082195 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:54.082200 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:54.083755 (+ 1555us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 20:36:54.083918 2267 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:54.079692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:41389 (local address 127.25.62.190:37437)
0619 20:36:54.079751 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:54.079753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:54.079761 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:54.079781 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:54.079782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:54.079789 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:54.079840 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:54.080089 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.080601 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.081527 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.081702 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.081755 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:54.083836 (+ 2081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.131:41389: BlockingRecv error: recv got EOF from 127.25.62.131:41389 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 20:36:54 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:54 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:54.087898 2531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:54.084078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.131:39695)
0619 20:36:54.084128 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:54.084139 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:54.084180 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:54.084357 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:54.084358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:54.084462 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:54.084486 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:54.084487 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.085087 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.085089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:54.085660 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:54.085661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.085960 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.085961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:54.085995 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:54.086398 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:54.086402 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:54.087861 (+ 1459us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 20:36:54.088019 2267 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:54.084137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:39695 (local address 127.25.62.190:37437)
0619 20:36:54.084237 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:54.084239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:54.084245 (+ 6us) server_negotiation.cc:407] Connection header received
0619 20:36:54.084260 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:54.084261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:54.084267 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:54.084314 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:54.084554 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.084954 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.085728 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.085892 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.085943 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:54.087948 (+ 2005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.131:39695: BlockingRecv error: recv got EOF from 127.25.62.131:39695 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260619 20:36:54.088040 2529 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 20:36:54 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:54 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:54.834477 2534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:54.827109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.129:58875)
0619 20:36:54.827251 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:54.827261 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:54.827312 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:54.827447 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:54.827448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:54.827601 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:54.827650 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:54.827652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.828334 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.828335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:54.828883 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:54.828885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.829262 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.829263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:54.829305 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:54.832666 (+ 3361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:54.832670 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:54.834330 (+ 1660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
W20260619 20:36:54.834527 2535 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:54.827189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:58875 (local address 127.25.62.190:37437)
0619 20:36:54.827294 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:54.827296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:54.827304 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:54.827370 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:54.827370 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:54.827380 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:54.827435 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:54.827727 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.828246 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.828962 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.829201 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.829255 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:54.834425 (+ 5170us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.129:58875: BlockingRecv error: recv got EOF from 127.25.62.129:58875 (error 108)
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":54,"threads_started":1}
Jun 19 20:36:54 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:54 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:54.969213 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:54.964280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.130:36899)
0619 20:36:54.964438 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:54.964450 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:54.964512 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:54.964674 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:54.964675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:54.964828 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:54.964876 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:54.964878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.965653 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.965654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:54.966209 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:54.966210 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.966574 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.966575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:54.966635 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:54.967205 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:54.967214 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:54.969115 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":67,"threads_started":1}
W20260619 20:36:54.969269 2535 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:54.964336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:36899 (local address 127.25.62.190:37437)
0619 20:36:54.964443 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:54.964445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:54.964498 (+ 53us) server_negotiation.cc:407] Connection header received
0619 20:36:54.964566 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:54.964567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:54.964577 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:54.964619 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:54.964978 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.965562 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.966289 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:54.966486 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:54.966542 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:54.969206 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.130:36899: BlockingRecv error: recv got EOF from 127.25.62.130:36899 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Jun 19 20:36:55 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:55 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:55.092710 2537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:55.088419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.131:36251)
0619 20:36:55.088582 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:55.088596 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:55.088671 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:55.088796 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:55.088797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:55.088945 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:55.088993 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:55.088995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:55.089662 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:55.089662 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:55.090192 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:55.090193 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:55.090508 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:55.090509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:55.090546 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:55.091019 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:55.091023 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:55.092650 (+ 1627us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":68,"threads_started":1}
W20260619 20:36:55.092823 2535 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:55.088492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:36251 (local address 127.25.62.190:37437)
0619 20:36:55.088565 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:55.088567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:55.088656 (+ 89us) server_negotiation.cc:407] Connection header received
0619 20:36:55.088715 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:55.088715 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:55.088727 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:55.088776 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:55.089087 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:55.089590 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:55.090262 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:55.090447 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:55.090497 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:55.092732 (+ 2235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.131:36251: BlockingRecv error: recv got EOF from 127.25.62.131:36251 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 19 20:36:55 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:55 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:55.839468 2538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:55.835004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.129:38539)
0619 20:36:55.835192 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:55.835204 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:55.835262 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:55.835390 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:55.835391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:55.835544 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:55.835594 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:55.835595 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:55.836252 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:55.836253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:55.836771 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:55.836772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:55.837149 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:55.837150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:55.837196 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:55.837608 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:55.837613 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:55.839381 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":95,"threads_started":1}
W20260619 20:36:55.839515 2539 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:55.835110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:38539 (local address 127.25.62.190:37437)
0619 20:36:55.835229 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:55.835231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:55.835244 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:36:55.835318 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:55.835319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:55.835328 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:55.835387 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:55.835653 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:55.836180 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:55.836838 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:55.837069 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:55.837148 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:55.839465 (+ 2317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.129:38539: BlockingRecv error: recv got EOF from 127.25.62.129:38539 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":71,"threads_started":1}
Jun 19 20:36:55 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:55 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:55.974630 2540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:55.969684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.130:59555)
0619 20:36:55.969868 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:55.969880 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:55.969985 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:55.970159 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:55.970160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:55.970339 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:55.970388 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:55.970390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:55.971110 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:55.971111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:55.971651 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:55.971653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:55.972051 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:55.972052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:55.972109 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:55.972610 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:55.972614 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:55.974555 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":100,"threads_started":1}
W20260619 20:36:55.974766 2539 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:55.969786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:59555 (local address 127.25.62.190:37437)
0619 20:36:55.969859 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:55.969861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:55.969964 (+ 103us) server_negotiation.cc:407] Connection header received
0619 20:36:55.970070 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:55.970071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:55.970090 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:55.970153 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:55.970480 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:55.970985 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:55.971740 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:55.971963 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:55.972051 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:55.974663 (+ 2612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.130:59555: BlockingRecv error: recv got EOF from 127.25.62.130:59555 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 19 20:36:56 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:56 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:56.097935 2541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:56.093182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.131:50673)
0619 20:36:56.093322 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:56.093342 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:56.093447 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:56.093624 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:56.093625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:56.093802 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:56.093848 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:56.093850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:56.094521 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:56.094521 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:56.095063 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:56.095064 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:56.095451 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:56.095452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:56.095514 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:56.096000 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:56.096004 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:56.097839 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":55,"threads_started":1}
W20260619 20:36:56.097987 2539 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:56.093334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:50673 (local address 127.25.62.190:37437)
0619 20:36:56.093392 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:56.093394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:56.093425 (+ 31us) server_negotiation.cc:407] Connection header received
0619 20:36:56.093532 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:56.093534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:56.093546 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:56.093617 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:56.093929 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:56.094424 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:56.095147 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:56.095341 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:56.095437 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:56.097922 (+ 2485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.131:50673: BlockingRecv error: recv got EOF from 127.25.62.131:50673 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Jun 19 20:36:56 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:56 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:56.845621 2542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:56.840025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.129:56585)
0619 20:36:56.840365 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:56.840377 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:56.840454 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:56.840608 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:56.840609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:56.840780 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:56.840834 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:56.840837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:56.841649 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:56.841650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:56.842500 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:56.842504 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:56.842990 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:56.842991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:56.843044 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:56.843588 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:56.843594 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:56.845544 (+ 1950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":267,"threads_started":1}
W20260619 20:36:56.845800 2543 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:56.840246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:56585 (local address 127.25.62.190:37437)
0619 20:36:56.840397 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:56.840399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:56.840438 (+ 39us) server_negotiation.cc:407] Connection header received
0619 20:36:56.840519 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:56.840520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:56.840531 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:56.840591 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:56.840966 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:56.841548 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:56.842612 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:56.842897 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:56.842977 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:56.845704 (+ 2727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.129:56585: BlockingRecv error: recv got EOF from 127.25.62.129:56585 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
Jun 19 20:36:56 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:56 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:56.979734 2544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:56.975102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.130:41809)
0619 20:36:56.975236 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:56.975248 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:56.975309 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:56.975419 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:56.975419 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:56.975605 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:56.975666 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:56.975668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:56.976331 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:56.976331 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:56.976946 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:56.976948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:56.977335 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:56.977336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:56.977377 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:56.977867 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:56.977871 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:56.979662 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
W20260619 20:36:56.979848 2543 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:56.975128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:41809 (local address 127.25.62.190:37437)
0619 20:36:56.975182 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:56.975184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:56.975297 (+ 113us) server_negotiation.cc:407] Connection header received
0619 20:36:56.975351 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:56.975352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:56.975363 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:56.975413 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:56.975729 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:56.976267 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:56.977034 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:56.977270 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:56.977328 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:56.979753 (+ 2425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.130:41809: BlockingRecv error: recv got EOF from 127.25.62.130:41809 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Jun 19 20:36:57 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:57 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:57.103256 2545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:57.098446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.131:36287)
0619 20:36:57.098629 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:57.098641 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:57.098715 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:57.098922 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:57.098923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:57.099105 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:57.099154 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:57.099155 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:57.099823 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:57.099824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:57.100376 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:57.100378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:57.100756 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:57.100757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:57.100814 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:57.101316 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:57.101321 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:57.103152 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":103,"threads_started":1}
W20260619 20:36:57.103288 2543 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:57.098544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:36287 (local address 127.25.62.190:37437)
0619 20:36:57.098615 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:57.098618 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:57.098694 (+ 76us) server_negotiation.cc:407] Connection header received
0619 20:36:57.098801 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:57.098802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:57.098823 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:57.098913 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:57.099234 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:57.099739 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:57.100465 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:57.100666 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:57.100754 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:57.103224 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.131:36287: BlockingRecv error: recv got EOF from 127.25.62.131:36287 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 19 20:36:57 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:57 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:57.850494 2546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:57.846149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.129:60323)
0619 20:36:57.846280 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:57.846290 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:57.846346 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:57.846461 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:57.846462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:57.846620 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:57.846669 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:57.846670 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:57.847315 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:57.847315 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:57.847835 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:57.847836 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:57.848166 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:57.848167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:57.848206 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:57.848702 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:57.848706 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:57.850437 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
W20260619 20:36:57.850613 2547 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:57.846206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:60323 (local address 127.25.62.190:37437)
0619 20:36:57.846341 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:57.846343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:57.846351 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:36:57.846388 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:57.846389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:57.846400 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:57.846457 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:57.846730 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:57.847246 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:57.847907 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:57.848104 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:57.848155 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:57.850533 (+ 2378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.129:60323: BlockingRecv error: recv got EOF from 127.25.62.129:60323 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
Jun 19 20:36:57 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:57 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:57.984505 2548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:57.980288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.130:46605)
0619 20:36:57.980412 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:57.980424 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:57.980495 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:57.980619 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:57.980619 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:57.980783 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:57.980835 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:57.980836 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:57.981499 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:57.981500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:57.982023 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:57.982025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:57.982344 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:57.982344 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:57.982391 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:57.982803 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:57.982807 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:57.984448 (+ 1641us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
W20260619 20:36:57.984606 2547 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:57.980352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:46605 (local address 127.25.62.190:37437)
0619 20:36:57.980422 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:57.980425 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:57.980491 (+ 66us) server_negotiation.cc:407] Connection header received
0619 20:36:57.980550 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:57.980550 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:57.980561 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:57.980613 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:57.980893 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:57.981408 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:57.982099 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:57.982280 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:57.982328 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:57.984539 (+ 2211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.130:46605: BlockingRecv error: recv got EOF from 127.25.62.130:46605 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 19 20:36:58 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:58 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:58.108484 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:58.103763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.131:34339)
0619 20:36:58.103941 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:58.103955 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:58.104046 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:58.104188 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:58.104188 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:58.104355 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:58.104401 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:58.104403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:58.105136 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:58.105137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:58.105670 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:58.105672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:58.106057 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:58.106058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:58.106124 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:58.106550 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:58.106554 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:58.108388 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":94,"threads_started":1}
W20260619 20:36:58.108520 2547 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:58.103849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:34339 (local address 127.25.62.190:37437)
0619 20:36:58.103945 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:58.103947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:58.104025 (+ 78us) server_negotiation.cc:407] Connection header received
0619 20:36:58.104116 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:58.104117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:58.104131 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:58.104172 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:58.104502 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:58.105011 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:58.105755 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:58.105971 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:58.106056 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:58.108460 (+ 2404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.131:34339: BlockingRecv error: recv got EOF from 127.25.62.131:34339 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 19 20:36:58 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:58 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:58.855453 2550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:58.851058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.129:38131)
0619 20:36:58.851207 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:58.851216 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:58.851265 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:58.851401 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:58.851401 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:58.851563 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:58.851612 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:58.851614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:58.852278 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:58.852279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:58.852794 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:58.852796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:58.853119 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:58.853119 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:58.853161 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:58.853579 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:58.853583 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:58.855369 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
W20260619 20:36:58.855623 2551 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:58.851133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:38131 (local address 127.25.62.190:37437)
0619 20:36:58.851264 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:58.851266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:58.851275 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:36:58.851312 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:58.851313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:58.851322 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:58.851385 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:58.851701 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:58.852208 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:58.852861 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:58.853066 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:58.853116 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:58.855508 (+ 2392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.129:38131: BlockingRecv error: recv got EOF from 127.25.62.129:38131 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":73,"threads_started":1}
Jun 19 20:36:58 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:58 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:58.989794 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:58.985203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.130:46449)
0619 20:36:58.985333 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:58.985346 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:58.985434 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:58.985577 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:58.985577 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:58.985750 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:58.985797 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:58.985799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:58.986507 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:58.986509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:58.987097 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:58.987099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:58.987424 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:58.987425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:58.987466 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:58.987909 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:58.987914 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:58.989736 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
W20260619 20:36:58.989948 2551 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:58.985319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:46449 (local address 127.25.62.190:37437)
0619 20:36:58.985361 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:58.985363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:58.985394 (+ 31us) server_negotiation.cc:407] Connection header received
0619 20:36:58.985504 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:58.985505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:58.985517 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:58.985573 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:58.985856 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:58.986442 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:58.987174 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:58.987358 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:58.987407 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:58.989850 (+ 2443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.130:46449: BlockingRecv error: recv got EOF from 127.25.62.130:46449 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Jun 19 20:36:59 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:59 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:59.114010 2553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:59.109020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.131:58095)
0619 20:36:59.109206 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:59.109218 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:59.109308 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:59.109460 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:59.109461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:59.109646 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:59.109693 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:59.109694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:59.110360 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:59.110361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:59.110898 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:59.110899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:59.111260 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:59.111261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:59.111318 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:59.111782 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:59.111786 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:59.113887 (+ 2101us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":119,"threads_started":1}
W20260619 20:36:59.114117 2551 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:59.109132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:58095 (local address 127.25.62.190:37437)
0619 20:36:59.109188 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:59.109190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:59.109288 (+ 98us) server_negotiation.cc:407] Connection header received
0619 20:36:59.109370 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:59.109371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:59.109385 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:59.109455 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:59.109770 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:59.110277 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:59.110992 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:59.111188 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:59.111262 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:59.114037 (+ 2775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.131:58095: BlockingRecv error: recv got EOF from 127.25.62.131:58095 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Jun 19 20:36:59 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:59 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:59.864681 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:59.856012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.129:34417)
0619 20:36:59.856194 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:59.856206 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:59.856271 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:59.856504 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:59.856506 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:59.856697 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:59.856758 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:59.856760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:59.857465 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:59.857465 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:59.857990 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:59.857991 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:59.858301 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:59.858302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:59.858356 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:59.862711 (+ 4355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:59.862717 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:59.864623 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":109,"threads_started":1}
W20260619 20:36:59.864861 2555 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:59.856047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:34417 (local address 127.25.62.190:37437)
0619 20:36:59.856198 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:59.856201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:59.856254 (+ 53us) server_negotiation.cc:407] Connection header received
0619 20:36:59.856358 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:59.856359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:59.856370 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:59.856454 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:59.856830 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:59.857398 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:59.858059 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:59.858239 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:59.858287 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:59.864752 (+ 6465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.129:34417: BlockingRecv error: recv got EOF from 127.25.62.129:34417 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
Jun 19 20:36:59 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:36:59 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:36:59.994982 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:36:59.990268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.130:46345)
0619 20:36:59.990404 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 20:36:59.990414 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:36:59.990474 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:36:59.990598 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:36:59.990599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:36:59.990773 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:36:59.990824 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:59.990825 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:59.991531 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:59.991532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:59.992047 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:36:59.992049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:36:59.992482 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:36:59.992483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:36:59.992523 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:59.993109 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:36:59.993114 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:36:59.994922 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
W20260619 20:36:59.995121 2555 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:36:59.990347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:46345 (local address 127.25.62.190:37437)
0619 20:36:59.990393 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:36:59.990394 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:36:59.990459 (+ 65us) server_negotiation.cc:407] Connection header received
0619 20:36:59.990519 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:36:59.990520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:36:59.990530 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:36:59.990582 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:36:59.990956 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:59.991454 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:59.992223 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:36:59.992411 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:36:59.992477 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:36:59.995034 (+ 2557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.130:46345: BlockingRecv error: recv got EOF from 127.25.62.130:46345 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Jun 19 20:37:00 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:00 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:00.119184 2557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:00.114428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.131:34009)
0619 20:37:00.114582 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:00.114604 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:00.114660 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:00.114795 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:00.114796 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:00.114965 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:00.115031 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:00.115033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:00.115704 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:00.115704 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:00.116221 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:00.116223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:00.116549 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:00.116550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:00.116601 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:00.117203 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:00.117208 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:00.119126 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
W20260619 20:37:00.119278 2555 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:00.114531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:34009 (local address 127.25.62.190:37437)
0619 20:37:00.114607 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:00.114608 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:00.114645 (+ 37us) server_negotiation.cc:407] Connection header received
0619 20:37:00.114702 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:00.114702 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:00.114714 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:00.114772 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:00.115129 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:00.115628 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:00.116299 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:00.116483 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:00.116538 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:00.119211 (+ 2673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.131:34009: BlockingRecv error: recv got EOF from 127.25.62.131:34009 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 19 20:37:00 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:00 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:00.869757 2558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:00.865218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.129:37137)
0619 20:37:00.865359 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:00.865369 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:00.865431 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:00.865560 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:00.865560 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:00.865717 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:00.865766 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:00.865768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:00.866451 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:00.866452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:00.866982 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:00.866984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:00.867359 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:00.867360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:00.867400 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:00.867825 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:00.867829 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:00.869699 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
W20260619 20:37:00.869875 2559 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:00.865277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:37137 (local address 127.25.62.190:37437)
0619 20:37:00.865409 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:00.865411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:00.865426 (+ 15us) server_negotiation.cc:407] Connection header received
0619 20:37:00.865492 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:00.865492 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:00.865503 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:00.865549 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:00.865824 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:00.866330 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:00.867075 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:00.867288 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:00.867347 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:00.869785 (+ 2438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.129:37137: BlockingRecv error: recv got EOF from 127.25.62.129:37137 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":88,"threads_started":1}
Jun 19 20:37:00 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:00 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:00.999884 2560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:00.995533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.130:58307)
0619 20:37:00.995703 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:00.995715 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:00.995782 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:00.995916 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:00.995916 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:00.996090 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:00.996148 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:00.996150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:00.996773 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:00.996774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:00.997367 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:00.997370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:00.997680 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:00.997681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:00.997720 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:00.998165 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:00.998168 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:00.999828 (+ 1660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":89,"threads_started":1}
W20260619 20:37:01.000005 2559 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:00.995581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:58307 (local address 127.25.62.190:37437)
0619 20:37:00.995636 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:00.995638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:00.995785 (+ 147us) server_negotiation.cc:407] Connection header received
0619 20:37:00.995842 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:00.995843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:00.995854 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:00.995898 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:00.996211 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:00.996709 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:00.997438 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:00.997618 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:00.997669 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:00.999926 (+ 2257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.130:58307: BlockingRecv error: recv got EOF from 127.25.62.130:58307 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Jun 19 20:37:01 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:01 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:01.124467 2561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:01.119652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.131:59647)
0619 20:37:01.119809 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:01.119820 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:01.119892 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:01.120039 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:01.120040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:01.120226 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:01.120275 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:01.120277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:01.120948 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:01.120949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:01.121517 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:01.121519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:01.121932 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:01.121933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:01.121995 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:01.122466 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:01.122470 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:01.124407 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
W20260619 20:37:01.124691 2559 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:01.119730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:59647 (local address 127.25.62.190:37437)
0619 20:37:01.119797 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:01.119800 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:01.119871 (+ 71us) server_negotiation.cc:407] Connection header received
0619 20:37:01.119946 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:01.119947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:01.119963 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:01.120033 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:01.120360 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:01.120853 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:01.121610 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:01.121804 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:01.121935 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:01.124628 (+ 2693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.131:59647: BlockingRecv error: recv got EOF from 127.25.62.131:59647 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 19 20:37:01 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:01 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:01.874938 2562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:01.870232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.129:42227)
0619 20:37:01.870363 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:01.870374 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:01.870421 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:01.870628 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:01.870628 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:01.870785 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:01.870834 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:01.870835 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:01.871523 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:01.871524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:01.872103 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:01.872105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:01.872520 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:01.872521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:01.872573 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:01.873117 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:01.873122 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:01.874878 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
W20260619 20:37:01.875111 2563 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:01.870377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:42227 (local address 127.25.62.190:37437)
0619 20:37:01.870513 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:01.870515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:01.870522 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:37:01.870543 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:01.870543 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:01.870553 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:01.870614 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:01.870911 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:01.871422 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:01.872210 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:01.872423 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:01.872492 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:01.875039 (+ 2547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.129:42227: BlockingRecv error: recv got EOF from 127.25.62.129:42227 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
Jun 19 20:37:01 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:01 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:02.005389 2564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:02.000393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.130:33059)
0619 20:37:02.000658 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:02.000670 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:02.000761 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:02.000943 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:02.000944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:02.001140 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:02.001188 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:02.001190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:02.002011 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:02.002012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:02.002549 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:02.002551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:02.002943 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:02.002944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:02.003010 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:02.003469 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:02.003473 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:02.005291 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":82,"threads_started":1}
W20260619 20:37:02.005450 2563 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:02.000471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:33059 (local address 127.25.62.190:37437)
0619 20:37:02.000654 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:02.000657 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:02.000743 (+ 86us) server_negotiation.cc:407] Connection header received
0619 20:37:02.000838 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:02.000839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:02.000853 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:02.000922 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:02.001269 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:02.001927 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:02.002636 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:02.002838 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:02.002944 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:02.005380 (+ 2436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.130:33059: BlockingRecv error: recv got EOF from 127.25.62.130:33059 (error 108)
Metrics: {"server-negotiator.queue_time_us":153}
Jun 19 20:37:02 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:02 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:02.130071 2565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:02.125016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.131:36357)
0619 20:37:02.125205 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:02.125219 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:02.125294 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:02.125435 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:02.125436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:02.125606 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:02.125658 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:02.125660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:02.126388 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:02.126390 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:02.127077 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:02.127081 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:02.127486 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:02.127487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:02.127558 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:02.128034 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:02.128039 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:02.129970 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":109,"threads_started":1}
W20260619 20:37:02.130127 2563 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:02.125219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:36357 (local address 127.25.62.190:37437)
0619 20:37:02.125288 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:02.125290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:02.125300 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:37:02.125348 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:02.125349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:02.125361 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:02.125411 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:02.125766 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:02.126292 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:02.127161 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:02.127400 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:02.127483 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:02.130060 (+ 2577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.131:36357: BlockingRecv error: recv got EOF from 127.25.62.131:36357 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 19 20:37:02 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:02 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:02.880013 2566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:02.875391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.129:56657)
0619 20:37:02.875529 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:02.875548 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:02.875597 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:02.875798 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:02.875798 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:02.875967 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:02.876018 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:02.876019 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:02.876660 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:02.876661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:02.877255 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:02.877257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:02.877592 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:02.877593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:02.877632 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:02.878070 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:02.878075 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:02.879920 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
W20260619 20:37:02.880082 2567 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:02.875502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:56657 (local address 127.25.62.190:37437)
0619 20:37:02.875629 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:02.875631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:02.875642 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:37:02.875688 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:02.875688 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:02.875703 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:02.875793 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:02.876087 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:02.876586 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:02.877329 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:02.877526 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:02.877583 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:02.880011 (+ 2428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.129:56657: BlockingRecv error: recv got EOF from 127.25.62.129:56657 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
Jun 19 20:37:03 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:03 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:03.010341 2568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:03.005886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.130:52727)
0619 20:37:03.006040 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:03.006050 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:03.006096 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:03.006226 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:03.006226 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:03.006382 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:03.006432 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:03.006433 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:03.007066 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:03.007066 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:03.007578 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:03.007579 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:03.007882 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:03.007883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:03.007927 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:03.008349 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:03.008354 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:03.010275 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
W20260619 20:37:03.010489 2567 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:03.006000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:52727 (local address 127.25.62.190:37437)
0619 20:37:03.006071 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:03.006072 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:03.006083 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:37:03.006154 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:03.006154 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:03.006164 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:03.006205 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:03.006497 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:03.007002 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:03.007644 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:03.007823 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:03.007884 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:03.010398 (+ 2514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.130:52727: BlockingRecv error: recv got EOF from 127.25.62.130:52727 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 19 20:37:03 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:03 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:03.135493 2569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:03.130608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.131:32823)
0619 20:37:03.130760 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:03.130769 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:03.130824 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:03.130988 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:03.130989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:03.131149 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:03.131199 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:03.131200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:03.131849 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:03.131850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:03.132411 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:03.132412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:03.132980 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:03.132981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:03.133102 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:03.133628 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:03.133633 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:03.135437 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":74,"threads_started":1}
W20260619 20:37:03.135643 2567 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:03.130704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:32823 (local address 127.25.62.190:37437)
0619 20:37:03.130755 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:03.130758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:03.130812 (+ 54us) server_negotiation.cc:407] Connection header received
0619 20:37:03.130900 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:03.130901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:03.130913 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:03.130957 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:03.131273 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:03.131767 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:03.132542 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:03.132883 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:03.132960 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:03.135556 (+ 2596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.131:32823: BlockingRecv error: recv got EOF from 127.25.62.131:32823 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Jun 19 20:37:03 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:03 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:03.885365 2570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:03.880587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.129:38411)
0619 20:37:03.880739 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:03.880751 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:03.880818 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:03.881016 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:03.881018 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:03.881205 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:03.881255 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:03.881256 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:03.882196 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:03.882197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:03.882744 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:03.882746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:03.883111 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:03.883112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:03.883160 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:03.883565 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:03.883569 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:03.885308 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
W20260619 20:37:03.885560 2571 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:03.880693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:38411 (local address 127.25.62.190:37437)
0619 20:37:03.880830 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:03.880831 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:03.880858 (+ 27us) server_negotiation.cc:407] Connection header received
0619 20:37:03.880894 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:03.880895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:03.880905 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:03.880973 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:03.881364 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:03.882106 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:03.882826 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:03.883030 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:03.883083 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:03.885465 (+ 2382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.129:38411: BlockingRecv error: recv got EOF from 127.25.62.129:38411 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
Jun 19 20:37:04 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:04 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:04.016116 2572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:04.010918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.130:38349)
0619 20:37:04.011042 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:04.011060 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:04.011133 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:04.011395 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:04.011397 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:04.011587 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:04.011637 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:04.011639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:04.012515 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:04.012516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:04.013135 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:04.013137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:04.013554 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:04.013555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:04.013610 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:04.014126 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:04.014131 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:04.016047 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":64,"threads_started":1}
W20260619 20:37:04.016222 2571 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:04.011002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:38349 (local address 127.25.62.190:37437)
0619 20:37:04.011055 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:04.011058 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:04.011116 (+ 58us) server_negotiation.cc:407] Connection header received
0619 20:37:04.011232 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:04.011233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:04.011244 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:04.011337 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:04.011752 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:04.012307 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:04.013255 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:04.013460 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:04.013524 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:04.016150 (+ 2626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.130:38349: BlockingRecv error: recv got EOF from 127.25.62.130:38349 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Jun 19 20:37:04 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:04 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:04.140563 2573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:04.135995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.131:59823)
0619 20:37:04.136166 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:04.136181 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:04.136245 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:04.136373 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:04.136374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:04.136545 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:04.136595 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:04.136596 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:04.137307 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:04.137308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:04.137828 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:04.137829 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:04.138174 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:04.138176 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:04.138229 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:04.138734 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:04.138739 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:04.140508 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":78,"threads_started":1}
W20260619 20:37:04.140769 2571 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:04.136156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:59823 (local address 127.25.62.190:37437)
0619 20:37:04.136220 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:04.136222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:04.136231 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:37:04.136291 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:04.136292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:04.136303 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:04.136351 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:04.136666 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:04.137239 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:04.137906 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:04.138088 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:04.138140 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:04.140703 (+ 2563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.131:59823: BlockingRecv error: recv got EOF from 127.25.62.131:59823 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Jun 19 20:37:04 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:04 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:04.893680 2574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:04.885825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.129:37007)
0619 20:37:04.885962 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:04.885974 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:04.886035 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:04.886185 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:04.886186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:04.886356 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:04.886405 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:04.886407 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:04.887166 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:04.887167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:04.887693 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:04.887695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:04.888022 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:04.888023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:04.888063 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:04.891794 (+ 3731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:04.891799 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:04.893609 (+ 1810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
W20260619 20:37:04.893775 2575 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:04.885892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:37007 (local address 127.25.62.190:37437)
0619 20:37:04.886011 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:04.886012 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:04.886022 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:37:04.886094 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:04.886095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:04.886106 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:04.886179 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:04.886475 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:04.887066 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:04.887768 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:04.887958 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:04.888010 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:04.893702 (+ 5692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.129:37007: BlockingRecv error: recv got EOF from 127.25.62.129:37007 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":67,"threads_started":1}
Jun 19 20:37:05 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:05 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:05.021576 2576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:05.016657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.130:38401)
0619 20:37:05.016877 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:05.016892 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:05.016993 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:05.017202 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:05.017203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:05.017406 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:05.017453 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:05.017455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:05.018257 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:05.018258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:05.018835 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:05.018837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:05.019178 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:05.019179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:05.019231 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:05.019705 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:05.019709 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:05.021511 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":101,"threads_started":1}
W20260619 20:37:05.021703 2575 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:05.016767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:38401 (local address 127.25.62.190:37437)
0619 20:37:05.016881 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:05.016883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:05.016972 (+ 89us) server_negotiation.cc:407] Connection header received
0619 20:37:05.017091 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:05.017092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:05.017106 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:05.017195 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:05.017537 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:05.018173 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:05.018908 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:05.019091 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:05.019145 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:05.021602 (+ 2457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.130:38401: BlockingRecv error: recv got EOF from 127.25.62.130:38401 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Jun 19 20:37:05 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:05 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:05.145503 2577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:05.141088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.131:55643)
0619 20:37:05.141237 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:05.141247 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:05.141303 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:05.141435 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:05.141436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:05.141601 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:05.141650 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:05.141651 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:05.142270 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:05.142271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:05.142791 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:05.142792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:05.143103 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:05.143103 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:05.143143 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:05.143556 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:05.143560 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:05.145434 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
W20260619 20:37:05.145571 2575 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:05.141139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:55643 (local address 127.25.62.190:37437)
0619 20:37:05.141197 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:05.141199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:05.141303 (+ 104us) server_negotiation.cc:407] Connection header received
0619 20:37:05.141361 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:05.141361 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:05.141373 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:05.141425 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:05.141710 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:05.142193 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:05.142862 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:05.143035 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:05.143082 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:05.145523 (+ 2441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.131:55643: BlockingRecv error: recv got EOF from 127.25.62.131:55643 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Jun 19 20:37:05 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:05 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:05.899122 2578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:05.894186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.129:37999)
0619 20:37:05.894374 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:05.894388 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:05.894481 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:05.894686 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:05.894688 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:05.894965 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:05.895014 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:05.895017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:05.895702 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:05.895703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:05.896262 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:05.896264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:05.896645 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:05.896646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:05.896704 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:05.897208 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:05.897213 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:05.899027 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":100,"threads_started":1}
W20260619 20:37:05.899170 2579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:05.894297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:37999 (local address 127.25.62.190:37437)
0619 20:37:05.894410 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:05.894411 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:05.894479 (+ 68us) server_negotiation.cc:407] Connection header received
0619 20:37:05.894533 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:05.894534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:05.894548 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:05.894676 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:05.895095 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:05.895616 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:05.896358 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:05.896559 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:05.896644 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:05.899102 (+ 2458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.129:37999: BlockingRecv error: recv got EOF from 127.25.62.129:37999 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":47,"threads_started":1}
Jun 19 20:37:06 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:06 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:06.026698 2580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:06.022057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.130:60287)
0619 20:37:06.022184 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:06.022197 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:06.022264 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:06.022419 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:06.022420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:06.022582 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:06.022629 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:06.022631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:06.023259 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:06.023260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:06.023791 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:06.023792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:06.024188 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:06.024189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:06.024230 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:06.024724 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:06.024732 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:06.026634 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":52,"threads_started":1}
W20260619 20:37:06.026834 2579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:06.022144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:60287 (local address 127.25.62.190:37437)
0619 20:37:06.022235 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:06.022236 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:06.022250 (+ 14us) server_negotiation.cc:407] Connection header received
0619 20:37:06.022326 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:06.022326 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:06.022338 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:06.022392 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:06.022691 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:06.023196 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:06.023904 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:06.024113 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:06.024183 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:06.026750 (+ 2567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.130:60287: BlockingRecv error: recv got EOF from 127.25.62.130:60287 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 19 20:37:06 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:06 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:06.150650 2581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:06.145996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.131:55935)
0619 20:37:06.146131 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:06.146146 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:06.146206 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:06.146342 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:06.146343 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:06.146519 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:06.146568 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:06.146569 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:06.147209 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:06.147210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:06.147739 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:06.147740 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:06.148100 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:06.148101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:06.148142 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:06.148645 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:06.148650 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:06.150589 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
W20260619 20:37:06.150754 2579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:06.146103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:55935 (local address 127.25.62.190:37437)
0619 20:37:06.146179 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:06.146181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:06.146193 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:37:06.146261 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:06.146262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:06.146274 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:06.146322 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:06.146639 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:06.147136 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:06.147833 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:06.148028 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:06.148093 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:06.150684 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.131:55935: BlockingRecv error: recv got EOF from 127.25.62.131:55935 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 19 20:37:06 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:06 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:06.904110 2582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:06.899628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.129:46615)
0619 20:37:06.899797 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:06.899807 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:06.899858 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:06.899999 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:06.899999 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:06.900187 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:06.900236 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:06.900238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:06.900880 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:06.900881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:06.901479 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:06.901482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:06.901820 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:06.901821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:06.901864 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:06.902323 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:06.902328 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:06.904051 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":95,"threads_started":1}
W20260619 20:37:06.904222 2583 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:06.899701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:46615 (local address 127.25.62.190:37437)
0619 20:37:06.899835 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:06.899836 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:06.899846 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:37:06.899908 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:06.899909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:06.899920 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:06.899982 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:06.900313 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:06.900811 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:06.901558 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:06.901752 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:06.901809 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:06.904142 (+ 2333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.129:46615: BlockingRecv error: recv got EOF from 127.25.62.129:46615 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
Jun 19 20:37:07 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:07 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:07.031778 2584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:07.027141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.130:55099)
0619 20:37:07.027309 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:07.027323 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:07.027398 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:07.027536 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:07.027537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:07.027701 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:07.027750 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:07.027751 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:07.028392 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:07.028393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:07.028941 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:07.028942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:07.029369 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:07.029370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:07.029411 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:07.029847 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:07.029851 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:07.031711 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":74,"threads_started":1}
W20260619 20:37:07.031893 2583 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:07.027303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:55099 (local address 127.25.62.190:37437)
0619 20:37:07.027382 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:07.027384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:07.027394 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:37:07.027464 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:07.027465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:07.027475 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:07.027518 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:07.027812 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:07.028312 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:07.029090 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:07.029295 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:07.029364 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:07.031807 (+ 2443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.130:55099: BlockingRecv error: recv got EOF from 127.25.62.130:55099 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 20:37:07 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:07 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:07.156144 2585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:07.151152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.131:46677)
0619 20:37:07.151422 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:07.151435 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:07.151531 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:07.151684 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:07.151685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:07.151870 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:07.151915 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:07.151917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:07.152585 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:07.152586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:07.153153 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:07.153155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:07.153542 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:07.153543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:07.153605 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:07.154097 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:07.154102 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:07.156050 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":79,"threads_started":1}
W20260619 20:37:07.156190 2583 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:07.151246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:46677 (local address 127.25.62.190:37437)
0619 20:37:07.151421 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:07.151424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:07.151511 (+ 87us) server_negotiation.cc:407] Connection header received
0619 20:37:07.151615 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:07.151616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:07.151630 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:07.151678 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:07.151997 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:07.152501 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:07.153242 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:07.153453 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:07.153541 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:07.156127 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.131:46677: BlockingRecv error: recv got EOF from 127.25.62.131:46677 (error 108)
Metrics: {"server-negotiator.queue_time_us":146}
Jun 19 20:37:07 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:07 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:07.909349 2586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:07.904659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.129:34395)
0619 20:37:07.904797 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:07.904812 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:07.904860 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:07.905015 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:07.905016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:07.905205 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:07.905253 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:07.905255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:07.905895 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:07.905895 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:07.906457 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:07.906460 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:07.906943 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:07.906945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:07.907008 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:07.907514 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:07.907519 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:07.909284 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
W20260619 20:37:07.909487 2587 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:07.904731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:34395 (local address 127.25.62.190:37437)
0619 20:37:07.904883 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:07.904885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:07.904893 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:37:07.904919 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:07.904920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:07.904933 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:07.904999 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:07.905318 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:07.905828 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:07.906545 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:07.906865 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:07.906933 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:07.909380 (+ 2447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.129:34395: BlockingRecv error: recv got EOF from 127.25.62.129:34395 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
Jun 19 20:37:08 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:08 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:08.036851 2588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:08.032295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.130:50705)
0619 20:37:08.032442 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:08.032459 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:08.032512 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:08.032660 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:08.032660 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:08.032825 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:08.032873 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:08.032875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:08.033522 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:08.033522 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:08.034042 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:08.034044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:08.034365 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:08.034366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:08.034404 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:08.034823 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:08.034827 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:08.036795 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
W20260619 20:37:08.037025 2587 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:08.032398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:50705 (local address 127.25.62.190:37437)
0619 20:37:08.032464 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:08.032466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:08.032500 (+ 34us) server_negotiation.cc:407] Connection header received
0619 20:37:08.032570 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:08.032571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:08.032581 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:08.032633 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:08.032934 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:08.033454 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:08.034109 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:08.034303 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:08.034353 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:08.036915 (+ 2562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.130:50705: BlockingRecv error: recv got EOF from 127.25.62.130:50705 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 20:37:08 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:08 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.131@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:08.160918 2589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:08.156643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.131:44005)
0619 20:37:08.156798 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:08.156809 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:08.156868 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:08.156978 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:08.156978 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:08.157157 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:08.157203 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:08.157204 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:08.157862 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:08.157862 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:08.158378 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:08.158379 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:08.158691 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:08.158692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:08.158729 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:08.159166 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:08.159170 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:08.160861 (+ 1691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":100,"threads_started":1}
W20260619 20:37:08.161018 2587 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:08.156699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:44005 (local address 127.25.62.190:37437)
0619 20:37:08.156761 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:08.156763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:08.156872 (+ 109us) server_negotiation.cc:407] Connection header received
0619 20:37:08.156911 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:08.156912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:08.156925 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:08.156972 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:08.157288 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:08.157793 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:08.158450 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:08.158630 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:08.158683 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:08.160951 (+ 2268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.131:44005: BlockingRecv error: recv got EOF from 127.25.62.131:44005 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Jun 19 20:37:08 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:08 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.129@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:08.914326 2590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:08.909876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.129:38641)
0619 20:37:08.910049 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:08.910059 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:08.910126 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:08.910255 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:08.910256 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:08.910414 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:08.910463 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:08.910464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:08.911091 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:08.911092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:08.911690 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:08.911692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:08.912017 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:08.912018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:08.912059 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:08.912575 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:08.912579 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:08.914269 (+ 1690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":117,"threads_started":1}
W20260619 20:37:08.914470 2591 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:08.909955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:38641 (local address 127.25.62.190:37437)
0619 20:37:08.910068 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:08.910070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:08.910127 (+ 57us) server_negotiation.cc:407] Connection header received
0619 20:37:08.910175 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:08.910176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:08.910186 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:08.910244 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:08.910520 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:08.911027 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:08.911754 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:08.911955 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:08.912008 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:08.914376 (+ 2368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.129:38641: BlockingRecv error: recv got EOF from 127.25.62.129:38641 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":45,"threads_started":1}
Jun 19 20:37:09 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 20:37:09 dist-test-slave-0t40 krb5kdc[1533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.25.62.130@KRBTEST.COM for oryx/127.25.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 20:37:09.042171 2592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:09.037429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37437 (local address 127.25.62.130:42519)
0619 20:37:09.037619 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:09.037630 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:09.037702 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:09.037825 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:09.037826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:09.038002 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:09.038055 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:09.038056 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:09.038892 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:09.038894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:09.039480 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:09.039482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:09.039881 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:09.039882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:09.039929 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:09.040353 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:09.040357 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:09.042104 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.25.62.190:37437: Server oryx/127.25.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":104,"threads_started":1}
W20260619 20:37:09.042367 2591 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:09.037518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:42519 (local address 127.25.62.190:37437)
0619 20:37:09.037571 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:09.037573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:09.037698 (+ 125us) server_negotiation.cc:407] Connection header received
0619 20:37:09.037761 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:09.037761 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:09.037772 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:09.037818 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:09.038180 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:09.038802 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:09.039557 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:09.039819 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:09.039875 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:09.042199 (+ 2324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.25.62.130:42519: BlockingRecv error: recv got EOF from 127.25.62.130:42519 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 20:37:09.079109 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 2116
I20260619 20:37:09.083778 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 2268
I20260619 20:37:09.087935 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 2400
I20260619 20:37:09.092033 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 2046
2026-06-19T20:37:09Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17411 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T20:37:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:37:09Z Disabled control of system clock
I20260619 20:37:09.119194 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:39091
--webserver_interface=127.25.62.190
--webserver_port=0
--builtin_ntp_servers=127.25.62.148:43663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:39091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 20:37:09.198740 2596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:37:09.198943 2596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:37:09.198964 2596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:37:09.200058 2596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:37:09.200094 2596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:37:09.200107 2596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:37:09.200119 2596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:37:09.200132 2596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:37:09.201392 2596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:43663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:39091
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:39091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:37:09.201570 2596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:37:09.201794 2596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:37:09.206420 2602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:09.206432 2601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:09.206470 2604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:09.206580 2596 server_base.cc:1034] running on GCE node
I20260619 20:37:09.206974 2596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:37:09.207279 2596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:37:09.208453 2596 hybrid_clock.cc:648] HybridClock initialized: now 1781901429208433 us; error 36 us; skew 500 ppm
I20260619 20:37:09.209756 2596 webserver.cc:460] Webserver started at http://127.25.62.190:39151/ using document root <none> and password file <none>
I20260619 20:37:09.210016 2596 fs_manager.cc:359] Metadata directory not provided
I20260619 20:37:09.210085 2596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:37:09.210235 2596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:37:09.211305 2596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "5abc142abc4a499da158a932e961e5a4"
format_stamp: "Formatted at 2026-06-19 20:37:09 on dist-test-slave-0t40"
server_key: "7f154f5c1ed5b6243eb4043b6d7abdbd"
server_key_iv: "60c1e4b81058420c8ae56ce07cb9fdd6"
server_key_version: "encryptionkey@0"
I20260619 20:37:09.211674 2596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "5abc142abc4a499da158a932e961e5a4"
format_stamp: "Formatted at 2026-06-19 20:37:09 on dist-test-slave-0t40"
server_key: "7f154f5c1ed5b6243eb4043b6d7abdbd"
server_key_iv: "60c1e4b81058420c8ae56ce07cb9fdd6"
server_key_version: "encryptionkey@0"
I20260619 20:37:09.212960 2596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:37:09.213719 2610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:09.213891 2596 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:37:09.213986 2596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "5abc142abc4a499da158a932e961e5a4"
format_stamp: "Formatted at 2026-06-19 20:37:09 on dist-test-slave-0t40"
server_key: "7f154f5c1ed5b6243eb4043b6d7abdbd"
server_key_iv: "60c1e4b81058420c8ae56ce07cb9fdd6"
server_key_version: "encryptionkey@0"
I20260619 20:37:09.214068 2596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:37:09.231885 2596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:37:09.232196 2596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:37:09.232352 2596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:37:09.236132 2596 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:39091
I20260619 20:37:09.236169 2662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:39091 every 8 connection(s)
I20260619 20:37:09.236522 2596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:37:09.237263 2663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:37:09.239722 2663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4: Bootstrap starting.
I20260619 20:37:09.240250 2663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4: Neither blocks nor log segments found. Creating new log.
I20260619 20:37:09.240477 2663 log.cc:824] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4: Log is configured to *not* fsync() on all Append() calls
I20260619 20:37:09.241185 2663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4: No bootstrap required, opened a new log
I20260619 20:37:09.242720 2663 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5abc142abc4a499da158a932e961e5a4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 39091 } }
I20260619 20:37:09.242849 2663 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:37:09.242870 2663 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5abc142abc4a499da158a932e961e5a4, State: Initialized, Role: FOLLOWER
I20260619 20:37:09.243010 2663 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5abc142abc4a499da158a932e961e5a4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 39091 } }
I20260619 20:37:09.243088 2663 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:37:09.243109 2663 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:37:09.243136 2663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:37:09.243367 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 2596
I20260619 20:37:09.243476 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:37:09.243660 25850 external_mini_cluster.cc:1428] Setting key 553f657634ff9c0e149e2e1147509797
I20260619 20:37:09.243850 2663 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5abc142abc4a499da158a932e961e5a4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 39091 } }
I20260619 20:37:09.243973 2663 leader_election.cc:302] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5abc142abc4a499da158a932e961e5a4; no voters:
I20260619 20:37:09.244331 2663 leader_election.cc:288] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:37:09.244374 2668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:37:09.244575 2668 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4 [term 1 LEADER]: Becoming Leader. State: Replica: 5abc142abc4a499da158a932e961e5a4, State: Running, Role: LEADER
I20260619 20:37:09.244673 2663 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:37:09.244717 2668 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5abc142abc4a499da158a932e961e5a4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 39091 } }
I20260619 20:37:09.245146 2670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5abc142abc4a499da158a932e961e5a4. Latest consensus state: current_term: 1 leader_uuid: "5abc142abc4a499da158a932e961e5a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5abc142abc4a499da158a932e961e5a4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 39091 } } }
I20260619 20:37:09.245239 2670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:37:09.245297 2669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5abc142abc4a499da158a932e961e5a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5abc142abc4a499da158a932e961e5a4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 39091 } } }
I20260619 20:37:09.245373 2669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:37:09.246174 2675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:37:09.246685 2675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:37:09.248099 2675 catalog_manager.cc:1347] Generated new cluster ID: ea052d8509334a28aeae2a7ac95f8b78
I20260619 20:37:09.248152 2675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:37:09.249778 2667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:09.244162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45746 (local address 127.25.62.190:39091)
0619 20:37:09.244419 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:09.244428 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:09.244463 (+ 35us) server_negotiation.cc:407] Connection header received
0619 20:37:09.244586 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:09.244594 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:09.244667 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:09.245100 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:09.245201 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:09.246614 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:09.247692 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:09.248127 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:09.248268 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:09.249162 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:37:09.249165 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:37:09.249166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:37:09.249333 (+ 167us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:37:09.249364 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:37:09.249375 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:37:09.249458 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:09.249532 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 20:37:09.249604 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":95,"threads_started":1}
I20260619 20:37:09.257048 2675 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:37:09.257745 2675 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:37:09.262123 2675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5abc142abc4a499da158a932e961e5a4: Generated new TSK 0
I20260619 20:37:09.262274 2675 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:37:09.265374 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:39091
--builtin_ntp_servers=127.25.62.148:43663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 20:37:09.363431 2687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:37:09.363628 2687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:37:09.363651 2687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:37:09.364830 2687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:37:09.364871 2687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:37:09.364895 2687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:37:09.366253 2687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:43663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:39091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:37:09.366492 2687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:37:09.366804 2687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:37:09.372611 2692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:09.372622 2693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:09.372638 2695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:09.373016 2687 server_base.cc:1034] running on GCE node
I20260619 20:37:09.373232 2687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:37:09.373471 2687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:37:09.374644 2687 hybrid_clock.cc:648] HybridClock initialized: now 1781901429374627 us; error 35 us; skew 500 ppm
I20260619 20:37:09.375851 2687 webserver.cc:460] Webserver started at http://127.25.62.129:37229/ using document root <none> and password file <none>
I20260619 20:37:09.376099 2687 fs_manager.cc:359] Metadata directory not provided
I20260619 20:37:09.376170 2687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:37:09.376327 2687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:37:09.377460 2687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "3ac84e9d52cb4f5b9075e9ea79241674"
format_stamp: "Formatted at 2026-06-19 20:37:09 on dist-test-slave-0t40"
server_key: "a9e50202b478335e6a6caa1bdf32a7d9"
server_key_iv: "950cd45b59841d6f21f3c56401da3439"
server_key_version: "encryptionkey@0"
I20260619 20:37:09.377805 2687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "3ac84e9d52cb4f5b9075e9ea79241674"
format_stamp: "Formatted at 2026-06-19 20:37:09 on dist-test-slave-0t40"
server_key: "a9e50202b478335e6a6caa1bdf32a7d9"
server_key_iv: "950cd45b59841d6f21f3c56401da3439"
server_key_version: "encryptionkey@0"
I20260619 20:37:09.379132 2687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:37:09.379904 2701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:09.380071 2687 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:37:09.380190 2687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "3ac84e9d52cb4f5b9075e9ea79241674"
format_stamp: "Formatted at 2026-06-19 20:37:09 on dist-test-slave-0t40"
server_key: "a9e50202b478335e6a6caa1bdf32a7d9"
server_key_iv: "950cd45b59841d6f21f3c56401da3439"
server_key_version: "encryptionkey@0"
I20260619 20:37:09.380262 2687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:37:09.400069 2687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:37:09.400395 2687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:37:09.400557 2687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:37:09.400782 2687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:37:09.401186 2687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:37:09.401244 2687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:09.401285 2687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:37:09.401314 2687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:09.407176 2687 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:44691
I20260619 20:37:09.407222 2814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:44691 every 8 connection(s)
I20260619 20:37:09.407547 2687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:37:09.410713 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 2687
I20260619 20:37:09.410813 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:37:09.411007 25850 external_mini_cluster.cc:1428] Setting key 83cf28289e52197440468031f5188df3
I20260619 20:37:09.411918 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:39091
--builtin_ntp_servers=127.25.62.148:43663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 20:37:09.413025 2667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:09.408196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:57695 (local address 127.25.62.190:39091)
0619 20:37:09.408263 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:09.408265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:09.408531 (+ 266us) server_negotiation.cc:407] Connection header received
0619 20:37:09.409096 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:09.409098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:09.409111 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:09.409167 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:09.409810 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:09.410319 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:09.411492 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:09.411751 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:09.411828 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:09.412768 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:37:09.412769 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:37:09.412769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:37:09.412774 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:37:09.412788 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:37:09.412793 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:37:09.412841 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:09.412909 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 20:37:09.412946 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 20:37:09.413338 2817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:09.408215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:39091 (local address 127.25.62.129:57695)
0619 20:37:09.408405 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:09.408439 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:09.408956 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:09.409238 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:09.409242 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:09.409250 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:09.409716 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:09.409720 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:09.410420 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:09.410421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:09.411408 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:09.411410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:09.411834 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:09.411835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:09.411982 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:09.412622 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:37:09.412622 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:09.412651 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:37:09.412673 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:37:09.412841 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:37:09.412846 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:37:09.412849 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:37:09.412914 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 20:37:09.413024 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
I20260619 20:37:09.413848 2815 heartbeater.cc:347] Connected to a master server at 127.25.62.190:39091
I20260619 20:37:09.413925 2815 heartbeater.cc:464] Registering TS with master...
I20260619 20:37:09.414113 2815 heartbeater.cc:511] Master 127.25.62.190:39091 requested a full tablet report, sending...
I20260619 20:37:09.414636 2627 ts_manager.cc:194] Registered new tserver with Master: 3ac84e9d52cb4f5b9075e9ea79241674 (127.25.62.129:44691)
I20260619 20:37:09.415570 2627 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.62.129:57695
W20260619 20:37:09.491736 2818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:37:09.492038 2818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:37:09.492069 2818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:37:09.493747 2818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:37:09.493841 2818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:37:09.493922 2818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:37:09.495409 2818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:43663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:39091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:37:09.495657 2818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:37:09.495968 2818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:37:09.500962 2826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:09.500965 2824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:09.501148 2818 server_base.cc:1034] running on GCE node
W20260619 20:37:09.500973 2823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:09.501493 2818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:37:09.501762 2818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:37:09.502923 2818 hybrid_clock.cc:648] HybridClock initialized: now 1781901429502907 us; error 28 us; skew 500 ppm
I20260619 20:37:09.504428 2818 webserver.cc:460] Webserver started at http://127.25.62.130:38635/ using document root <none> and password file <none>
I20260619 20:37:09.504709 2818 fs_manager.cc:359] Metadata directory not provided
I20260619 20:37:09.504786 2818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:37:09.504976 2818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:37:09.506129 2818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "7ebee4ddfd7d4e6fa5f57d5b9cc3668b"
format_stamp: "Formatted at 2026-06-19 20:37:09 on dist-test-slave-0t40"
server_key: "20ae0e0277496427a68b6ebafe99862b"
server_key_iv: "99912125b4802ea92714316c0843ffa7"
server_key_version: "encryptionkey@0"
I20260619 20:37:09.506570 2818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "7ebee4ddfd7d4e6fa5f57d5b9cc3668b"
format_stamp: "Formatted at 2026-06-19 20:37:09 on dist-test-slave-0t40"
server_key: "20ae0e0277496427a68b6ebafe99862b"
server_key_iv: "99912125b4802ea92714316c0843ffa7"
server_key_version: "encryptionkey@0"
I20260619 20:37:09.507977 2818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:37:09.508782 2832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:09.509016 2818 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:37:09.509152 2818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "7ebee4ddfd7d4e6fa5f57d5b9cc3668b"
format_stamp: "Formatted at 2026-06-19 20:37:09 on dist-test-slave-0t40"
server_key: "20ae0e0277496427a68b6ebafe99862b"
server_key_iv: "99912125b4802ea92714316c0843ffa7"
server_key_version: "encryptionkey@0"
I20260619 20:37:09.509246 2818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:37:09.525404 2818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:37:09.525723 2818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:37:09.525887 2818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:37:09.526120 2818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:37:09.526468 2818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:37:09.526530 2818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:09.526577 2818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:37:09.526623 2818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:09.532864 2818 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:42699
I20260619 20:37:09.532902 2945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:42699 every 8 connection(s)
I20260619 20:37:09.533288 2818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:37:09.537104 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 2818
I20260619 20:37:09.537221 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:37:09.537415 25850 external_mini_cluster.cc:1428] Setting key 0a8424285d634e0d8ca14490d4b3ac01
I20260619 20:37:09.538467 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:39091
--builtin_ntp_servers=127.25.62.148:43663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 20:37:09.538828 2667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:09.534097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:41355 (local address 127.25.62.190:39091)
0619 20:37:09.534190 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:09.534192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:09.534385 (+ 193us) server_negotiation.cc:407] Connection header received
0619 20:37:09.534922 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:09.534923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:09.534934 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:09.534994 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:09.535678 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:09.536193 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:09.537270 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:09.537471 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:09.537558 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:09.538526 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:37:09.538528 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:37:09.538528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:37:09.538533 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:37:09.538550 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:37:09.538556 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:37:09.538616 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:09.538744 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 20:37:09.538767 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 20:37:09.539117 2948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:09.534043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:39091 (local address 127.25.62.130:41355)
0619 20:37:09.534254 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:09.534290 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:09.534788 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:09.535072 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:09.535075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:09.535083 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:09.535586 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:09.535591 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:09.536289 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:09.536290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:09.537170 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:09.537173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:09.537569 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:09.537570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:09.537700 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:09.538390 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:37:09.538390 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:09.538417 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:37:09.538440 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:37:09.538644 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:37:09.538650 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:37:09.538653 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:37:09.538721 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 20:37:09.538833 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260619 20:37:09.539664 2946 heartbeater.cc:347] Connected to a master server at 127.25.62.190:39091
I20260619 20:37:09.539767 2946 heartbeater.cc:464] Registering TS with master...
I20260619 20:37:09.539969 2946 heartbeater.cc:511] Master 127.25.62.190:39091 requested a full tablet report, sending...
I20260619 20:37:09.540364 2627 ts_manager.cc:194] Registered new tserver with Master: 7ebee4ddfd7d4e6fa5f57d5b9cc3668b (127.25.62.130:42699)
I20260619 20:37:09.540776 2627 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.62.130:41355
W20260619 20:37:09.618351 2949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:37:09.618546 2949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:37:09.618566 2949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:37:09.619608 2949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:37:09.619642 2949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:37:09.619666 2949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:37:09.620997 2949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:43663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:39091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:37:09.621229 2949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:37:09.621464 2949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:37:09.626127 2955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:09.626123 2957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:09.626184 2954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:09.626574 2949 server_base.cc:1034] running on GCE node
I20260619 20:37:09.626726 2949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:37:09.626914 2949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:37:09.628050 2949 hybrid_clock.cc:648] HybridClock initialized: now 1781901429628036 us; error 24 us; skew 500 ppm
I20260619 20:37:09.629215 2949 webserver.cc:460] Webserver started at http://127.25.62.131:45795/ using document root <none> and password file <none>
I20260619 20:37:09.629398 2949 fs_manager.cc:359] Metadata directory not provided
I20260619 20:37:09.629438 2949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:37:09.629523 2949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:37:09.630460 2949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "2e2be90d41674278a9a36eab53a3ef8b"
format_stamp: "Formatted at 2026-06-19 20:37:09 on dist-test-slave-0t40"
server_key: "aa12b257ea90e15c0d7f502f7bcd9d43"
server_key_iv: "56b051d2e1197c9bf2be318446d8c893"
server_key_version: "encryptionkey@0"
I20260619 20:37:09.630744 2949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "2e2be90d41674278a9a36eab53a3ef8b"
format_stamp: "Formatted at 2026-06-19 20:37:09 on dist-test-slave-0t40"
server_key: "aa12b257ea90e15c0d7f502f7bcd9d43"
server_key_iv: "56b051d2e1197c9bf2be318446d8c893"
server_key_version: "encryptionkey@0"
I20260619 20:37:09.631911 2949 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:37:09.632530 2963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:09.632704 2949 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:37:09.632800 2949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "2e2be90d41674278a9a36eab53a3ef8b"
format_stamp: "Formatted at 2026-06-19 20:37:09 on dist-test-slave-0t40"
server_key: "aa12b257ea90e15c0d7f502f7bcd9d43"
server_key_iv: "56b051d2e1197c9bf2be318446d8c893"
server_key_version: "encryptionkey@0"
I20260619 20:37:09.632893 2949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:37:09.640475 2949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:37:09.640744 2949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:37:09.640895 2949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:37:09.641161 2949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:37:09.641477 2949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:37:09.641531 2949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:09.641572 2949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:37:09.641598 2949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:09.647787 2949 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:34263
I20260619 20:37:09.647822 3076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:34263 every 8 connection(s)
I20260619 20:37:09.648147 2949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:37:09.653131 2667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:09.648848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:34707 (local address 127.25.62.190:39091)
0619 20:37:09.648919 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:09.648921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:09.649204 (+ 283us) server_negotiation.cc:407] Connection header received
0619 20:37:09.649665 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:09.649666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:09.649676 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:09.649730 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:09.650360 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:09.650864 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:09.651785 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:09.651978 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:09.652031 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:09.652822 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:37:09.652823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:37:09.652823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:37:09.652828 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:37:09.652841 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:37:09.652844 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:37:09.652902 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:09.652977 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 20:37:09.653000 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 20:37:09.653360 3079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:09.648841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:39091 (local address 127.25.62.131:34707)
0619 20:37:09.649028 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:09.649109 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:09.649570 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:09.649778 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:09.649781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:09.649789 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:09.650250 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:09.650254 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:09.650950 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:09.650951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:09.651707 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:09.651709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:09.652073 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:09.652075 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:09.652197 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:09.652686 (+ 489us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:37:09.652687 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:09.652714 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:37:09.652735 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:37:09.652903 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:37:09.652907 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:37:09.652910 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:37:09.652963 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:37:09.653106 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260619 20:37:09.653404 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 2949
I20260619 20:37:09.653501 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:37:09.653692 25850 external_mini_cluster.cc:1428] Setting key 8038987dc0bacb7627557a0551e7b769
I20260619 20:37:09.653955 3077 heartbeater.cc:347] Connected to a master server at 127.25.62.190:39091
I20260619 20:37:09.654156 3077 heartbeater.cc:464] Registering TS with master...
I20260619 20:37:09.654358 3077 heartbeater.cc:511] Master 127.25.62.190:39091 requested a full tablet report, sending...
I20260619 20:37:09.654697 2627 ts_manager.cc:194] Registered new tserver with Master: 2e2be90d41674278a9a36eab53a3ef8b (127.25.62.131:34263)
I20260619 20:37:09.655051 2627 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.62.131:34707
I20260619 20:37:09.665864 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 20:37:09.667865 2667 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:09.667335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45748 (local address 127.25.62.190:39091)
0619 20:37:09.667397 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:09.667399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:09.667551 (+ 152us) server_negotiation.cc:407] Connection header received
0619 20:37:09.667607 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:09.667608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:09.667619 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:09.667671 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:09.667794 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45748: BlockingRecv error: recv got EOF from 127.0.0.1:45748 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:37:09.667975 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 2687
I20260619 20:37:09.672343 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 2818
I20260619 20:37:09.676399 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 2949
I20260619 20:37:09.680837 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 2596
2026-06-19T20:37:09Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (587 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T20:37:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:37:09Z Disabled control of system clock
I20260619 20:37:09.706638 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:37839
--webserver_interface=127.25.62.190
--webserver_port=0
--builtin_ntp_servers=127.25.62.148:40395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:37839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 20:37:09.786599 3088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:37:09.786799 3088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:37:09.786821 3088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:37:09.787961 3088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:37:09.788007 3088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:37:09.788022 3088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:37:09.788038 3088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:37:09.788051 3088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:37:09.789427 3088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:40395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:37839
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:37839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:37:09.789680 3088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:37:09.789978 3088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:37:09.794994 3093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:09.795077 3088 server_base.cc:1034] running on GCE node
W20260619 20:37:09.794972 3094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:09.794971 3096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:09.795473 3088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:37:09.795758 3088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:37:09.796902 3088 hybrid_clock.cc:648] HybridClock initialized: now 1781901429796892 us; error 31 us; skew 500 ppm
I20260619 20:37:09.798177 3088 webserver.cc:460] Webserver started at http://127.25.62.190:38497/ using document root <none> and password file <none>
I20260619 20:37:09.798424 3088 fs_manager.cc:359] Metadata directory not provided
I20260619 20:37:09.798470 3088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:37:09.798636 3088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:37:09.799666 3088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "6203a342452a40f2bfe3bfe65000d537"
format_stamp: "Formatted at 2026-06-19 20:37:09 on dist-test-slave-0t40"
server_key: "3a68e32de29170bcc44d55fea71b6253"
server_key_iv: "170dbbb370cc6d03758d63985d0e54ac"
server_key_version: "encryptionkey@0"
I20260619 20:37:09.800017 3088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "6203a342452a40f2bfe3bfe65000d537"
format_stamp: "Formatted at 2026-06-19 20:37:09 on dist-test-slave-0t40"
server_key: "3a68e32de29170bcc44d55fea71b6253"
server_key_iv: "170dbbb370cc6d03758d63985d0e54ac"
server_key_version: "encryptionkey@0"
I20260619 20:37:09.801329 3088 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:37:09.802033 3102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:09.802232 3088 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:37:09.802346 3088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "6203a342452a40f2bfe3bfe65000d537"
format_stamp: "Formatted at 2026-06-19 20:37:09 on dist-test-slave-0t40"
server_key: "3a68e32de29170bcc44d55fea71b6253"
server_key_iv: "170dbbb370cc6d03758d63985d0e54ac"
server_key_version: "encryptionkey@0"
I20260619 20:37:09.802428 3088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:37:09.805802 3088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:37:09.806063 3088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:37:09.806206 3088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:37:09.810236 3088 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:37839
I20260619 20:37:09.810268 3154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:37839 every 8 connection(s)
I20260619 20:37:09.810590 3088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:37:09.811297 3155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:37:09.814116 3155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537: Bootstrap starting.
I20260619 20:37:09.814667 3155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537: Neither blocks nor log segments found. Creating new log.
I20260619 20:37:09.814919 3155 log.cc:824] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537: Log is configured to *not* fsync() on all Append() calls
I20260619 20:37:09.815593 3155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537: No bootstrap required, opened a new log
I20260619 20:37:09.817171 3155 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6203a342452a40f2bfe3bfe65000d537" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37839 } }
I20260619 20:37:09.817308 3155 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:37:09.817329 3155 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6203a342452a40f2bfe3bfe65000d537, State: Initialized, Role: FOLLOWER
I20260619 20:37:09.817466 3155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6203a342452a40f2bfe3bfe65000d537" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37839 } }
I20260619 20:37:09.817541 3155 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:37:09.817564 3155 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:37:09.817592 3155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:37:09.818090 3155 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6203a342452a40f2bfe3bfe65000d537" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37839 } }
I20260619 20:37:09.818176 3155 leader_election.cc:302] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6203a342452a40f2bfe3bfe65000d537; no voters:
I20260619 20:37:09.818291 3155 leader_election.cc:288] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:37:09.818390 3158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:37:09.818580 3155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:37:09.818612 3158 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537 [term 1 LEADER]: Becoming Leader. State: Replica: 6203a342452a40f2bfe3bfe65000d537, State: Running, Role: LEADER
I20260619 20:37:09.818697 3158 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6203a342452a40f2bfe3bfe65000d537" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37839 } }
I20260619 20:37:09.819175 3160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6203a342452a40f2bfe3bfe65000d537. Latest consensus state: current_term: 1 leader_uuid: "6203a342452a40f2bfe3bfe65000d537" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6203a342452a40f2bfe3bfe65000d537" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37839 } } }
I20260619 20:37:09.819262 3160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537 [sys.catalog]: This master's current role is: LEADER
I20260619 20:37:09.819383 3159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6203a342452a40f2bfe3bfe65000d537" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6203a342452a40f2bfe3bfe65000d537" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37839 } } }
I20260619 20:37:09.819473 3159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537 [sys.catalog]: This master's current role is: LEADER
I20260619 20:37:09.819514 3167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:37:09.820041 3167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:37:09.820648 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 3088
I20260619 20:37:09.820752 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:37:09.820950 25850 external_mini_cluster.cc:1428] Setting key 1042c907c8bb5a96ee677fd48d314879
I20260619 20:37:09.822068 3167 catalog_manager.cc:1347] Generated new cluster ID: 46a7050f1d35419c8029032c700fd3d7
I20260619 20:37:09.822117 3167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:37:09.823002 3177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:09.821388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33234 (local address 127.25.62.190:37839)
0619 20:37:09.821595 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:09.821606 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:09.821682 (+ 76us) server_negotiation.cc:407] Connection header received
0619 20:37:09.821786 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:09.821791 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:09.821824 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:09.821975 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:09.822583 (+ 608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:37:09.822586 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:37:09.822586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:37:09.822590 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:37:09.822644 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:37:09.822655 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:37:09.822726 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:09.822797 (+ 71us) server_negotiation.cc:299] Negotiation successful
0619 20:37:09.822852 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
I20260619 20:37:09.829962 3167 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:37:09.830140 3167 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:37:09.834091 3167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6203a342452a40f2bfe3bfe65000d537: Generated new TSK 0
I20260619 20:37:09.834254 3167 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:37:09.837509 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:37839
--builtin_ntp_servers=127.25.62.148:40395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 20:37:09.936483 3179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:37:09.936717 3179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:37:09.936756 3179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:37:09.937928 3179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:37:09.937994 3179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:37:09.938037 3179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:37:09.939355 3179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:40395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:37839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:37:09.939615 3179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:37:09.939909 3179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:37:09.945839 3184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:09.945822 3187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:09.945829 3185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:09.946410 3179 server_base.cc:1034] running on GCE node
I20260619 20:37:09.946655 3179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:37:09.946900 3179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:37:09.948063 3179 hybrid_clock.cc:648] HybridClock initialized: now 1781901429948035 us; error 40 us; skew 500 ppm
I20260619 20:37:09.954154 3179 webserver.cc:460] Webserver started at http://127.25.62.129:44805/ using document root <none> and password file <none>
I20260619 20:37:09.954391 3179 fs_manager.cc:359] Metadata directory not provided
I20260619 20:37:09.954473 3179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:37:09.954638 3179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:37:09.955703 3179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "baf30b4366bf4561b1c45bbd3ac9990f"
format_stamp: "Formatted at 2026-06-19 20:37:09 on dist-test-slave-0t40"
server_key: "b5f9b7335b0013ccbbe1596b3f12532f"
server_key_iv: "c105601e1383f1e56171418e2a68a2be"
server_key_version: "encryptionkey@0"
I20260619 20:37:09.956070 3179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "baf30b4366bf4561b1c45bbd3ac9990f"
format_stamp: "Formatted at 2026-06-19 20:37:09 on dist-test-slave-0t40"
server_key: "b5f9b7335b0013ccbbe1596b3f12532f"
server_key_iv: "c105601e1383f1e56171418e2a68a2be"
server_key_version: "encryptionkey@0"
I20260619 20:37:09.957425 3179 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:37:09.958195 3193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:09.958405 3179 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:37:09.958518 3179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "baf30b4366bf4561b1c45bbd3ac9990f"
format_stamp: "Formatted at 2026-06-19 20:37:09 on dist-test-slave-0t40"
server_key: "b5f9b7335b0013ccbbe1596b3f12532f"
server_key_iv: "c105601e1383f1e56171418e2a68a2be"
server_key_version: "encryptionkey@0"
I20260619 20:37:09.958597 3179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:37:09.963388 3179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:37:09.963671 3179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:37:09.963804 3179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:37:09.964002 3179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:37:09.964311 3179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:37:09.964347 3179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:09.964375 3179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:37:09.964399 3179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:09.971185 3179 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:34583
I20260619 20:37:09.971273 3306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:34583 every 8 connection(s)
I20260619 20:37:09.971592 3179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:37:09.972918 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 3179
I20260619 20:37:09.973008 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:37:09.973248 25850 external_mini_cluster.cc:1428] Setting key 9fd39d19712a39e691cb734115387905
I20260619 20:37:09.974717 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:37839
--builtin_ntp_servers=127.25.62.148:40395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 20:37:09.975225 3177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:09.972328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:53019 (local address 127.25.62.190:37839)
0619 20:37:09.972369 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:09.972371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:09.972663 (+ 292us) server_negotiation.cc:407] Connection header received
0619 20:37:09.973288 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:09.973289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:09.973297 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:09.973353 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:09.974969 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:37:09.974970 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:37:09.974971 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:37:09.974974 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:37:09.974989 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:37:09.974993 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:37:09.975061 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:09.975139 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 20:37:09.975156 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:37:09.975484 3309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:09.972320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37839 (local address 127.25.62.129:53019)
0619 20:37:09.972508 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:09.972566 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:09.973180 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:09.973560 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:09.973564 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:09.973575 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:09.974670 (+ 1095us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:37:09.974671 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:09.974698 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:37:09.974725 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:37:09.975060 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:37:09.975067 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:37:09.975070 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:37:09.975140 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 20:37:09.975201 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260619 20:37:09.975939 3307 heartbeater.cc:347] Connected to a master server at 127.25.62.190:37839
I20260619 20:37:09.976027 3307 heartbeater.cc:464] Registering TS with master...
I20260619 20:37:09.976173 3307 heartbeater.cc:511] Master 127.25.62.190:37839 requested a full tablet report, sending...
I20260619 20:37:09.976689 3119 ts_manager.cc:194] Registered new tserver with Master: baf30b4366bf4561b1c45bbd3ac9990f (127.25.62.129:34583)
W20260619 20:37:10.054435 3310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:37:10.054692 3310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:37:10.054729 3310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:37:10.055898 3310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:37:10.055966 3310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:37:10.056010 3310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:37:10.057360 3310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:40395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:37839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:37:10.057633 3310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:37:10.057922 3310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:37:10.063179 3310 server_base.cc:1034] running on GCE node
W20260619 20:37:10.063119 3315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:10.063112 3318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:10.063119 3316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:10.063690 3310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:37:10.063930 3310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:37:10.065095 3310 hybrid_clock.cc:648] HybridClock initialized: now 1781901430065082 us; error 29 us; skew 500 ppm
I20260619 20:37:10.066450 3310 webserver.cc:460] Webserver started at http://127.25.62.130:43059/ using document root <none> and password file <none>
I20260619 20:37:10.066704 3310 fs_manager.cc:359] Metadata directory not provided
I20260619 20:37:10.066752 3310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:37:10.066927 3310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:37:10.067967 3310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "a9b72c8559b646a197883f4445d332e5"
format_stamp: "Formatted at 2026-06-19 20:37:10 on dist-test-slave-0t40"
server_key: "1756f8bd5cab6b5065c59345bf4b363d"
server_key_iv: "c0404d63ba8cbdb68e9b9e7b7d6f8dab"
server_key_version: "encryptionkey@0"
I20260619 20:37:10.068316 3310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "a9b72c8559b646a197883f4445d332e5"
format_stamp: "Formatted at 2026-06-19 20:37:10 on dist-test-slave-0t40"
server_key: "1756f8bd5cab6b5065c59345bf4b363d"
server_key_iv: "c0404d63ba8cbdb68e9b9e7b7d6f8dab"
server_key_version: "encryptionkey@0"
I20260619 20:37:10.069667 3310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:37:10.070346 3324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:10.070523 3310 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:37:10.070609 3310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "a9b72c8559b646a197883f4445d332e5"
format_stamp: "Formatted at 2026-06-19 20:37:10 on dist-test-slave-0t40"
server_key: "1756f8bd5cab6b5065c59345bf4b363d"
server_key_iv: "c0404d63ba8cbdb68e9b9e7b7d6f8dab"
server_key_version: "encryptionkey@0"
I20260619 20:37:10.070683 3310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:37:10.074018 3310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:37:10.074270 3310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:37:10.074400 3310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:37:10.074622 3310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:37:10.074925 3310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:37:10.074961 3310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:10.075016 3310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:37:10.075044 3310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:10.080655 3310 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:42865
I20260619 20:37:10.080732 3437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:42865 every 8 connection(s)
I20260619 20:37:10.081107 3310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:37:10.084029 3177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:10.081789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:58025 (local address 127.25.62.190:37839)
0619 20:37:10.081836 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:10.081838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:10.082053 (+ 215us) server_negotiation.cc:407] Connection header received
0619 20:37:10.082539 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:10.082540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:10.082546 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:10.082603 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:10.083779 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:37:10.083781 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:37:10.083781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:37:10.083786 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:37:10.083803 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:37:10.083808 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:37:10.083869 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:10.083944 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 20:37:10.083959 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:37:10.084311 3440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:10.081763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37839 (local address 127.25.62.130:58025)
0619 20:37:10.081929 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:10.081967 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:10.082434 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:10.082675 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:10.082679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:10.082688 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:10.083629 (+ 941us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:37:10.083630 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:10.083656 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:37:10.083684 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:37:10.083868 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:37:10.083873 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:37:10.083876 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:37:10.083955 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 20:37:10.084014 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":65,"threads_started":1}
I20260619 20:37:10.084738 3438 heartbeater.cc:347] Connected to a master server at 127.25.62.190:37839
I20260619 20:37:10.084824 3438 heartbeater.cc:464] Registering TS with master...
I20260619 20:37:10.084939 3438 heartbeater.cc:511] Master 127.25.62.190:37839 requested a full tablet report, sending...
I20260619 20:37:10.085315 3119 ts_manager.cc:194] Registered new tserver with Master: a9b72c8559b646a197883f4445d332e5 (127.25.62.130:42865)
I20260619 20:37:10.090055 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 3310
I20260619 20:37:10.090132 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:37:10.090276 25850 external_mini_cluster.cc:1428] Setting key 3d7cd2977681417a4fefb96f95611c17
I20260619 20:37:10.091126 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:37839
--builtin_ntp_servers=127.25.62.148:40395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 20:37:10.171243 3441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:37:10.171464 3441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:37:10.171502 3441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:37:10.172626 3441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:37:10.172686 3441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:37:10.172724 3441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:37:10.174083 3441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:40395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:37839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:37:10.174311 3441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:37:10.174546 3441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:37:10.179525 3447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:10.179533 3446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:10.179580 3441 server_base.cc:1034] running on GCE node
W20260619 20:37:10.179534 3449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:10.180006 3441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:37:10.180253 3441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:37:10.181416 3441 hybrid_clock.cc:648] HybridClock initialized: now 1781901430181403 us; error 30 us; skew 500 ppm
I20260619 20:37:10.182684 3441 webserver.cc:460] Webserver started at http://127.25.62.131:35675/ using document root <none> and password file <none>
I20260619 20:37:10.182930 3441 fs_manager.cc:359] Metadata directory not provided
I20260619 20:37:10.182999 3441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:37:10.183149 3441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:37:10.184155 3441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "3c92f384487e48058b2ae27251b67532"
format_stamp: "Formatted at 2026-06-19 20:37:10 on dist-test-slave-0t40"
server_key: "8a581091debc55180a616f7a15698231"
server_key_iv: "675c9c3de41cba11a6e00babece479b9"
server_key_version: "encryptionkey@0"
I20260619 20:37:10.184505 3441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "3c92f384487e48058b2ae27251b67532"
format_stamp: "Formatted at 2026-06-19 20:37:10 on dist-test-slave-0t40"
server_key: "8a581091debc55180a616f7a15698231"
server_key_iv: "675c9c3de41cba11a6e00babece479b9"
server_key_version: "encryptionkey@0"
I20260619 20:37:10.185851 3441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:37:10.186518 3455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:10.186687 3441 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:37:10.186779 3441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "3c92f384487e48058b2ae27251b67532"
format_stamp: "Formatted at 2026-06-19 20:37:10 on dist-test-slave-0t40"
server_key: "8a581091debc55180a616f7a15698231"
server_key_iv: "675c9c3de41cba11a6e00babece479b9"
server_key_version: "encryptionkey@0"
I20260619 20:37:10.186853 3441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:37:10.190106 3441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:37:10.190346 3441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:37:10.190475 3441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:37:10.190675 3441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:37:10.190965 3441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:37:10.191015 3441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:10.191053 3441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:37:10.191080 3441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:10.196497 3441 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:36195
I20260619 20:37:10.196538 3568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:36195 every 8 connection(s)
I20260619 20:37:10.196892 3441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:37:10.199867 3177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:10.197638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:42815 (local address 127.25.62.190:37839)
0619 20:37:10.197702 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:10.197704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:10.197921 (+ 217us) server_negotiation.cc:407] Connection header received
0619 20:37:10.198416 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:10.198418 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:10.198425 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:10.198474 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:10.199591 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:37:10.199592 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:37:10.199593 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:37:10.199597 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:37:10.199613 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:37:10.199617 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:37:10.199679 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:10.199780 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 20:37:10.199796 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:37:10.200068 3571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:10.197622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37839 (local address 127.25.62.131:42815)
0619 20:37:10.197802 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:10.197833 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:10.198295 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:10.198543 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:10.198551 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:10.198559 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:10.199433 (+ 874us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:37:10.199434 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:10.199461 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:37:10.199488 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:37:10.199686 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:37:10.199690 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:37:10.199693 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:37:10.199788 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 20:37:10.199848 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260619 20:37:10.200548 3569 heartbeater.cc:347] Connected to a master server at 127.25.62.190:37839
I20260619 20:37:10.200623 3569 heartbeater.cc:464] Registering TS with master...
I20260619 20:37:10.200778 3569 heartbeater.cc:511] Master 127.25.62.190:37839 requested a full tablet report, sending...
I20260619 20:37:10.201220 3119 ts_manager.cc:194] Registered new tserver with Master: 3c92f384487e48058b2ae27251b67532 (127.25.62.131:36195)
I20260619 20:37:10.205586 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 3441
I20260619 20:37:10.205742 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:37:10.205968 25850 external_mini_cluster.cc:1428] Setting key a0723abbf4967f32204b45503f43a81b
I20260619 20:37:10.206875 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 20:37:10.208884 3177 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:37:10.208289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33242 (local address 127.25.62.190:37839)
0619 20:37:10.208387 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:10.208390 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:10.208483 (+ 93us) server_negotiation.cc:407] Connection header received
0619 20:37:10.208604 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:10.208605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:10.208612 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:10.208683 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:10.208817 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33242: BlockingRecv error: recv got EOF from 127.0.0.1:33242 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 20:37:10.208989 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 3179
I20260619 20:37:10.213552 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 3310
I20260619 20:37:10.217897 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 3441
I20260619 20:37:10.222064 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 3088
2026-06-19T20:37:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (541 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:37:10 dist-test-slave-0t40 krb5kdc[3579](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:37:10 dist-test-slave-0t40 krb5kdc[3579](info): set up 2 sockets
Jun 19 20:37:10 dist-test-slave-0t40 krb5kdc[3579](info): commencing operation
krb5kdc: starting...
W20260619 20:37:11.262279 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:37:11 dist-test-slave-0t40 krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:37:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:37:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:37:11.381915 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:44673
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:46269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:44673
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:37:11.459961 3592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:37:11.460150 3592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:37:11.460171 3592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:37:11.461349 3592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:37:11.461387 3592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:37:11.461402 3592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:37:11.461416 3592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:37:11.461462 3592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:37:11.462862 3592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:46269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:44673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:44673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:37:11.463097 3592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:37:11.463317 3592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:37:11.468236 3592 server_base.cc:1034] running on GCE node
W20260619 20:37:11.468219 3598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:11.468210 3597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:11.468201 3600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:11.468672 3592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:37:11.468954 3592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:37:11.470098 3592 hybrid_clock.cc:648] HybridClock initialized: now 1781901431470094 us; error 37 us; skew 500 ppm
Jun 19 20:37:11 dist-test-slave-0t40 krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901431, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:37:11.472527 3592 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:37:11.472981 3592 webserver.cc:460] Webserver started at http://127.25.62.190:34707/ using document root <none> and password file <none>
I20260619 20:37:11.473227 3592 fs_manager.cc:359] Metadata directory not provided
I20260619 20:37:11.473294 3592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:37:11.473444 3592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:37:11.474423 3592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "877bc2df5ce740b2909708a6f9442439"
format_stamp: "Formatted at 2026-06-19 20:37:11 on dist-test-slave-0t40"
server_key: "ce9dec1c72bf6e7d9f5b2d55b2689a33"
server_key_iv: "91d8b6a1b8443a82a765dba6dbe25122"
server_key_version: "encryptionkey@0"
I20260619 20:37:11.474797 3592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "877bc2df5ce740b2909708a6f9442439"
format_stamp: "Formatted at 2026-06-19 20:37:11 on dist-test-slave-0t40"
server_key: "ce9dec1c72bf6e7d9f5b2d55b2689a33"
server_key_iv: "91d8b6a1b8443a82a765dba6dbe25122"
server_key_version: "encryptionkey@0"
I20260619 20:37:11.476078 3592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:37:11.476758 3607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:11.476930 3592 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:37:11.477029 3592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "877bc2df5ce740b2909708a6f9442439"
format_stamp: "Formatted at 2026-06-19 20:37:11 on dist-test-slave-0t40"
server_key: "ce9dec1c72bf6e7d9f5b2d55b2689a33"
server_key_iv: "91d8b6a1b8443a82a765dba6dbe25122"
server_key_version: "encryptionkey@0"
I20260619 20:37:11.477142 3592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:37:11.489266 3592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:37:11.491509 3592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:37:11.491669 3592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:37:11.495747 3592 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:44673
I20260619 20:37:11.495815 3659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:44673 every 8 connection(s)
I20260619 20:37:11.496170 3592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:37:11.497218 3660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:37:11.499959 3660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439: Bootstrap starting.
I20260619 20:37:11.500720 3660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439: Neither blocks nor log segments found. Creating new log.
I20260619 20:37:11.501075 3660 log.cc:824] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439: Log is configured to *not* fsync() on all Append() calls
I20260619 20:37:11.501871 3660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439: No bootstrap required, opened a new log
I20260619 20:37:11.504014 3660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877bc2df5ce740b2909708a6f9442439" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44673 } }
I20260619 20:37:11.504168 3660 raft_consensus.cc:374] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:37:11.504195 3660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 877bc2df5ce740b2909708a6f9442439, State: Initialized, Role: FOLLOWER
I20260619 20:37:11.504323 3660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877bc2df5ce740b2909708a6f9442439" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44673 } }
I20260619 20:37:11.504374 3660 raft_consensus.cc:388] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:37:11.504413 3660 raft_consensus.cc:482] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:37:11.504447 3660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:37:11.505488 3660 raft_consensus.cc:504] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877bc2df5ce740b2909708a6f9442439" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44673 } }
I20260619 20:37:11.505582 3660 leader_election.cc:302] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 877bc2df5ce740b2909708a6f9442439; no voters:
I20260619 20:37:11.505728 3660 leader_election.cc:288] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:37:11.505755 3663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:37:11.505864 3663 raft_consensus.cc:686] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439 [term 1 LEADER]: Becoming Leader. State: Replica: 877bc2df5ce740b2909708a6f9442439, State: Running, Role: LEADER
I20260619 20:37:11.505970 3660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:37:11.505951 3663 consensus_queue.cc:237] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877bc2df5ce740b2909708a6f9442439" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44673 } }
I20260619 20:37:11.506255 3663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "877bc2df5ce740b2909708a6f9442439" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877bc2df5ce740b2909708a6f9442439" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44673 } } }
I20260619 20:37:11.506312 3663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439 [sys.catalog]: This master's current role is: LEADER
I20260619 20:37:11.506420 3663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 877bc2df5ce740b2909708a6f9442439. Latest consensus state: current_term: 1 leader_uuid: "877bc2df5ce740b2909708a6f9442439" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877bc2df5ce740b2909708a6f9442439" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44673 } } }
I20260619 20:37:11.506476 3663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439 [sys.catalog]: This master's current role is: LEADER
I20260619 20:37:11.507418 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 3592
I20260619 20:37:11.507519 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:37:11.507741 25850 external_mini_cluster.cc:1428] Setting key e4b7c63658954457b571077f9842b019
W20260619 20:37:11.508030 3678 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:37:11.508095 3678 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:37:11.508149 3668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:37:11.508308 3668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:37:11.510491 3668 catalog_manager.cc:1347] Generated new cluster ID: 20f4d7dbc59d47fcbcccc24f4fe16ce0
I20260619 20:37:11.510547 3668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:37:11 dist-test-slave-0t40 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:37:11.518097 3668 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:37:11.518858 3668 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:37:11.520082 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:11.508410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53288 (local address 127.25.62.190:44673)
0619 20:37:11.508589 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:11.508598 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:11.508879 (+ 281us) server_negotiation.cc:407] Connection header received
0619 20:37:11.509127 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:11.509135 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:11.509192 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:11.509299 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:11.510129 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:11.511512 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:11.512581 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:11.513229 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:11.513327 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:11.516217 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:37:11.516221 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:37:11.516222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:37:11.516235 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:37:11.518559 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:11.518951 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:11.518955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:11.518955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:11.519046 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:11.519267 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:11.519267 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:11.519268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:11.519483 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:37:11.519658 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:11.519726 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 20:37:11.519879 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260619 20:37:11.526476 3668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439: Generated new TSK 0
I20260619 20:37:11.526646 3668 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
I20260619 20:37:11.559947 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:44673
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:46269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:37:11.655063 3686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:37:11.655292 3686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:37:11.655331 3686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:37:11.656514 3686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:37:11.656579 3686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:37:11.656622 3686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:37:11.658124 3686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:46269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:44673
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:37:11.658396 3686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:37:11.658617 3686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:37:11.663774 3694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:11.663801 3691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:11.663813 3692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:11.664018 3686 server_base.cc:1034] running on GCE node
I20260619 20:37:11.664245 3686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:37:11.664542 3686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:37:11.665704 3686 hybrid_clock.cc:648] HybridClock initialized: now 1781901431665687 us; error 33 us; skew 500 ppm
Jun 19 20:37:11 dist-test-slave-0t40 krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901431, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:37:11.668190 3686 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:37:11.668665 3686 webserver.cc:460] Webserver started at http://127.25.62.129:36155/ using document root <none> and password file <none>
I20260619 20:37:11.668912 3686 fs_manager.cc:359] Metadata directory not provided
I20260619 20:37:11.668983 3686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:37:11.669153 3686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:37:11.670161 3686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "f932d7fd27e645f396cd1fa339170d56"
format_stamp: "Formatted at 2026-06-19 20:37:11 on dist-test-slave-0t40"
server_key: "64671f1bca5759ccfbe3093a4fa49fde"
server_key_iv: "f5d18db162bbd5766c8eed78d9e67cc9"
server_key_version: "encryptionkey@0"
I20260619 20:37:11.670514 3686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "f932d7fd27e645f396cd1fa339170d56"
format_stamp: "Formatted at 2026-06-19 20:37:11 on dist-test-slave-0t40"
server_key: "64671f1bca5759ccfbe3093a4fa49fde"
server_key_iv: "f5d18db162bbd5766c8eed78d9e67cc9"
server_key_version: "encryptionkey@0"
I20260619 20:37:11.671859 3686 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:37:11.672655 3701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:11.672958 3686 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:37:11.673080 3686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "f932d7fd27e645f396cd1fa339170d56"
format_stamp: "Formatted at 2026-06-19 20:37:11 on dist-test-slave-0t40"
server_key: "64671f1bca5759ccfbe3093a4fa49fde"
server_key_iv: "f5d18db162bbd5766c8eed78d9e67cc9"
server_key_version: "encryptionkey@0"
I20260619 20:37:11.673157 3686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:37:11.683207 3686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:37:11.684909 3686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:37:11.685011 3686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:37:11.685297 3686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:37:11.685599 3686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:37:11.685657 3686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:11.685698 3686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:37:11.685725 3686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:11.692214 3686 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:34455
I20260619 20:37:11.692262 3814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:34455 every 8 connection(s)
I20260619 20:37:11.692643 3686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:37:11.694994 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 3686
I20260619 20:37:11.695102 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:37:11.695308 25850 external_mini_cluster.cc:1428] Setting key 4e4d3531e07d73e6d1c92310658eb5f4
Jun 19 20:37:11 dist-test-slave-0t40 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901431, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.62.130@KRBTEST.COM; defaulting to no policy
I20260619 20:37:11.703735 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:11.693344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:42295 (local address 127.25.62.190:44673)
0619 20:37:11.693416 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:11.693419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:11.693691 (+ 272us) server_negotiation.cc:407] Connection header received
0619 20:37:11.694365 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:11.694366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:11.694378 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:11.694442 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:11.695735 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:11.696460 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:11.698109 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:11.698398 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:11.698454 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:11.700749 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:37:11.700750 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:37:11.700751 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:37:11.700758 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:37:11.702367 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:11.702782 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:11.702784 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:11.702785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:11.702845 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:11.703088 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:11.703090 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:11.703090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:11.703206 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:37:11.703277 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:11.703561 (+ 284us) server_negotiation.cc:299] Negotiation successful
0619 20:37:11.703645 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 20:37:11.703938 3817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:11.693314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:44673 (local address 127.25.62.129:42295)
0619 20:37:11.693532 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:11.693581 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:11.694239 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:11.694527 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:11.694532 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:11.695332 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:11.695649 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:11.695656 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:11.696548 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:11.696549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:11.698000 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:11.698004 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:11.698487 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:11.698488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:11.698633 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:11.699103 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:11.699113 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:11.700641 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:37:11.702465 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:37:11.702472 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:37:11.702474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:37:11.702693 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:37:11.702917 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:37:11.702918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:37:11.702918 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:37:11.703021 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:37:11.703282 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:37:11.703288 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:37:11.703432 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 20:37:11.703539 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 20:37:11.703690 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":112,"threads_started":1}
I20260619 20:37:11.704459 3815 heartbeater.cc:347] Connected to a master server at 127.25.62.190:44673
I20260619 20:37:11.704568 3815 heartbeater.cc:464] Registering TS with master...
I20260619 20:37:11.704735 3815 heartbeater.cc:511] Master 127.25.62.190:44673 requested a full tablet report, sending...
I20260619 20:37:11.705274 3624 ts_manager.cc:194] Registered new tserver with Master: f932d7fd27e645f396cd1fa339170d56 (127.25.62.129:34455)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.130@KRBTEST.COM" created.
I20260619 20:37:11.705909 3624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:42295
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
I20260619 20:37:11.719950 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:44673
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:46269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:37:11.798533 3820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:37:11.798771 3820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:37:11.798810 3820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:37:11.799963 3820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:37:11.800033 3820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:37:11.800079 3820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:37:11.801582 3820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:46269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:44673
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:37:11.801834 3820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:37:11.802059 3820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:37:11.806993 3826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:11.806990 3828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:11.806998 3825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:11.807123 3820 server_base.cc:1034] running on GCE node
I20260619 20:37:11.807390 3820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:37:11.807621 3820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:37:11.808794 3820 hybrid_clock.cc:648] HybridClock initialized: now 1781901431808792 us; error 36 us; skew 500 ppm
Jun 19 20:37:11 dist-test-slave-0t40 krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901431, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:37:11.811213 3820 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:37:11.811678 3820 webserver.cc:460] Webserver started at http://127.25.62.130:46041/ using document root <none> and password file <none>
I20260619 20:37:11.811901 3820 fs_manager.cc:359] Metadata directory not provided
I20260619 20:37:11.811967 3820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:37:11.812120 3820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:37:11.813206 3820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "40fd0cd8d53a40dea61c996bfa488508"
format_stamp: "Formatted at 2026-06-19 20:37:11 on dist-test-slave-0t40"
server_key: "f659e2cac3d3581430b96c752eaebdc4"
server_key_iv: "8e39e3c7b90f0cb402487d704e2409bb"
server_key_version: "encryptionkey@0"
I20260619 20:37:11.813551 3820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "40fd0cd8d53a40dea61c996bfa488508"
format_stamp: "Formatted at 2026-06-19 20:37:11 on dist-test-slave-0t40"
server_key: "f659e2cac3d3581430b96c752eaebdc4"
server_key_iv: "8e39e3c7b90f0cb402487d704e2409bb"
server_key_version: "encryptionkey@0"
I20260619 20:37:11.814826 3820 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:37:11.815503 3835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:11.815668 3820 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:37:11.815762 3820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "40fd0cd8d53a40dea61c996bfa488508"
format_stamp: "Formatted at 2026-06-19 20:37:11 on dist-test-slave-0t40"
server_key: "f659e2cac3d3581430b96c752eaebdc4"
server_key_iv: "8e39e3c7b90f0cb402487d704e2409bb"
server_key_version: "encryptionkey@0"
I20260619 20:37:11.815840 3820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:37:11.825237 3820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:37:11.826920 3820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:37:11.827085 3820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:37:11.827296 3820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:37:11.827581 3820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:37:11.827636 3820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:11.827669 3820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:37:11.827697 3820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:11.833385 3820 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:33013
I20260619 20:37:11.833428 3948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:33013 every 8 connection(s)
I20260619 20:37:11.833719 3820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:37:11.834323 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 3820
I20260619 20:37:11.834429 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:37:11.834671 25850 external_mini_cluster.cc:1428] Setting key dc73c8e0e9f9723e1a93465f048497ee
Jun 19 20:37:11 dist-test-slave-0t40 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901431, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.62.131@KRBTEST.COM; defaulting to no policy
I20260619 20:37:11.843669 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:11.834476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:50771 (local address 127.25.62.190:44673)
0619 20:37:11.834544 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:11.834546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:11.834783 (+ 237us) server_negotiation.cc:407] Connection header received
0619 20:37:11.835305 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:11.835307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:11.835320 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:11.835361 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:11.836269 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:11.837093 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:11.838027 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:11.838358 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:11.838436 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:11.840697 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:37:11.840699 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:37:11.840699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:37:11.840708 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:37:11.842358 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:11.842783 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:11.842785 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:11.842786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:11.842845 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:11.843074 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:11.843074 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:11.843075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:11.843228 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:37:11.843299 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:11.843535 (+ 236us) server_negotiation.cc:299] Negotiation successful
0619 20:37:11.843607 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 20:37:11.843781 3951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:11.834443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:44673 (local address 127.25.62.130:50771)
0619 20:37:11.834653 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:11.834700 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:11.835190 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:11.835438 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:11.835442 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:11.835995 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:11.836171 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:11.836175 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:11.837175 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:11.837176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:11.837942 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:11.837944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:11.838444 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:11.838445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:11.838547 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:11.839048 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:11.839055 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:11.840603 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:37:11.842473 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:37:11.842481 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:37:11.842483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:37:11.842705 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:37:11.842918 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:37:11.842919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:37:11.842919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:37:11.843010 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:37:11.843297 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:37:11.843301 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:37:11.843409 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 20:37:11.843487 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 20:37:11.843609 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":112,"threads_started":1}
I20260619 20:37:11.844290 3949 heartbeater.cc:347] Connected to a master server at 127.25.62.190:44673
I20260619 20:37:11.844400 3949 heartbeater.cc:464] Registering TS with master...
I20260619 20:37:11.844561 3949 heartbeater.cc:511] Master 127.25.62.190:44673 requested a full tablet report, sending...
I20260619 20:37:11.844898 3624 ts_manager.cc:194] Registered new tserver with Master: 40fd0cd8d53a40dea61c996bfa488508 (127.25.62.130:33013)
I20260619 20:37:11.845355 3624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:50771
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
I20260619 20:37:11.860561 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:44673
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:46269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:37:11.939359 3954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:37:11.939550 3954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:37:11.939572 3954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:37:11.940745 3954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:37:11.940783 3954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:37:11.940809 3954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:37:11.942338 3954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:46269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:44673
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:37:11.942556 3954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:37:11.942778 3954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:37:11.947707 3959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:11.947707 3962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:11.947955 3954 server_base.cc:1034] running on GCE node
W20260619 20:37:11.947962 3960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:11.948235 3954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:37:11.948469 3954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:37:11.949621 3954 hybrid_clock.cc:648] HybridClock initialized: now 1781901431949616 us; error 35 us; skew 500 ppm
Jun 19 20:37:11 dist-test-slave-0t40 krb5kdc[3579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901431, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:37:11.951905 3954 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:37:11.952363 3954 webserver.cc:460] Webserver started at http://127.25.62.131:41297/ using document root <none> and password file <none>
I20260619 20:37:11.952582 3954 fs_manager.cc:359] Metadata directory not provided
I20260619 20:37:11.952647 3954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:37:11.952804 3954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:37:11.953850 3954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "e387c43bc0314bd79a5feef8377915e8"
format_stamp: "Formatted at 2026-06-19 20:37:11 on dist-test-slave-0t40"
server_key: "9235b8a20974a838091bd0310dcfef85"
server_key_iv: "76bd653ac4c8d189d51ad18a049b8782"
server_key_version: "encryptionkey@0"
I20260619 20:37:11.954202 3954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "e387c43bc0314bd79a5feef8377915e8"
format_stamp: "Formatted at 2026-06-19 20:37:11 on dist-test-slave-0t40"
server_key: "9235b8a20974a838091bd0310dcfef85"
server_key_iv: "76bd653ac4c8d189d51ad18a049b8782"
server_key_version: "encryptionkey@0"
I20260619 20:37:11.955492 3954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:37:11.956197 3969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:11.956367 3954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:37:11.956457 3954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "e387c43bc0314bd79a5feef8377915e8"
format_stamp: "Formatted at 2026-06-19 20:37:11 on dist-test-slave-0t40"
server_key: "9235b8a20974a838091bd0310dcfef85"
server_key_iv: "76bd653ac4c8d189d51ad18a049b8782"
server_key_version: "encryptionkey@0"
I20260619 20:37:11.956540 3954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:37:11.966876 3954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:37:11.968664 3954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:37:11.968818 3954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:37:11.969122 3954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:37:11.969422 3954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:37:11.969473 3954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:11.969518 3954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:37:11.969547 3954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:11.975135 3954 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:46739
I20260619 20:37:11.975204 4082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:46739 every 8 connection(s)
I20260619 20:37:11.975517 3954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:37:11 dist-test-slave-0t40 krb5kdc[3579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901431, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:37:11.984671 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:11.976232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:56739 (local address 127.25.62.190:44673)
0619 20:37:11.976294 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:11.976296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:11.976568 (+ 272us) server_negotiation.cc:407] Connection header received
0619 20:37:11.977077 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:11.977078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:11.977089 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:11.977159 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:11.978033 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:11.978527 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:11.979524 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:11.979730 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:11.979826 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:11.981985 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:37:11.981987 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:37:11.981987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:37:11.981994 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:37:11.983475 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:11.983892 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:11.983894 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:11.983894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:11.983952 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:11.984166 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:11.984167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:11.984167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:11.984267 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:37:11.984317 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:11.984545 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 20:37:11.984608 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:37:11.984861 4085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:11.976184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:44673 (local address 127.25.62.131:56739)
0619 20:37:11.976336 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:11.976375 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:11.976953 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:11.977213 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:11.977216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:11.977751 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:11.977925 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:11.977930 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:11.978618 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:11.978619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:11.979419 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:11.979421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:11.979827 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:11.979828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:11.979922 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:11.980431 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:11.980439 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:11.981886 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:37:11.983562 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:37:11.983567 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:37:11.983568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:37:11.983791 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:37:11.984019 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:37:11.984020 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:37:11.984020 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:37:11.984112 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:37:11.984320 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:37:11.984323 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:37:11.984439 (+ 116us) client_negotiation.cc:769] Sending connection context
0619 20:37:11.984519 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 20:37:11.984631 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260619 20:37:11.985028 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 3954
I20260619 20:37:11.985146 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:37:11.985409 25850 external_mini_cluster.cc:1428] Setting key b81f9288235e82122331fa1b27e5c5af
I20260619 20:37:11.985410 4083 heartbeater.cc:347] Connected to a master server at 127.25.62.190:44673
I20260619 20:37:11.985495 4083 heartbeater.cc:464] Registering TS with master...
I20260619 20:37:11.985682 4083 heartbeater.cc:511] Master 127.25.62.190:44673 requested a full tablet report, sending...
I20260619 20:37:11.986143 3623 ts_manager.cc:194] Registered new tserver with Master: e387c43bc0314bd79a5feef8377915e8 (127.25.62.131:46739)
I20260619 20:37:11.986531 3623 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:56739
I20260619 20:37:11.997090 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:37:12.004335 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:11.998473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53300 (local address 127.25.62.190:44673)
0619 20:37:11.998550 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:11.998552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:11.998638 (+ 86us) server_negotiation.cc:407] Connection header received
0619 20:37:11.998721 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:11.998722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:11.998734 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:11.998788 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:11.999346 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:11.999829 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.000550 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.000744 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.000826 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:12.002055 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:37:12.002056 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:37:12.002056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:37:12.002062 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:37:12.003301 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:12.003656 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:12.003656 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:12.003657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:12.003711 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:12.003875 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:12.003876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:12.003876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:12.004011 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:37:12.004089 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:12.004174 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 20:37:12.004233 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 20:37:12.006349 3623 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:37:12.008158 3623 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:37:12.014370 4092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:12.010195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:33013 (local address 127.0.0.1:47926)
0619 20:37:12.010364 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:12.010403 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:12.010468 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:12.010666 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:12.010669 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:12.010671 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:37:12.010738 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:12.010741 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.012071 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.012072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:12.013345 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:12.013348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.014021 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.014022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:12.014115 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:12.014129 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 20:37:12.014199 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 20:37:12.014254 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":44,"thread_start_us":57,"threads_started":1}
I20260619 20:37:12.014464 4094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:12.010295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47926 (local address 127.25.62.130:33013)
0619 20:37:12.010484 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:12.010489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:12.010499 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:37:12.010545 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:12.010547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:12.010563 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:12.010655 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:37:12.010831 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.011975 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.013448 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.013920 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.014017 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:12.014078 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:12.014335 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 20:37:12.014382 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":84,"threads_started":1}
I20260619 20:37:12.014639 4095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:12.010196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:46739 (local address 127.0.0.1:35584)
0619 20:37:12.010488 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:12.010500 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:12.010554 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:12.010847 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:12.010848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:12.010850 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:37:12.010923 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:12.010924 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.012273 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.012274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:12.013164 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:12.013167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.014116 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.014117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:12.014188 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:12.014192 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:37:12.014238 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 20:37:12.014281 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":4,"thread_start_us":55,"threads_started":1}
I20260619 20:37:12.014642 4096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:12.010602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51564 (local address 127.25.62.129:34455)
0619 20:37:12.010774 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:12.010779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:12.010792 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:37:12.010867 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:12.010869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:12.010884 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:12.010964 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:37:12.011150 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.011860 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.013443 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.014126 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.014192 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:12.014240 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:12.014335 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 20:37:12.014376 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":74,"threads_started":1}
I20260619 20:37:12.014882 4091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:12.010393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:34455 (local address 127.0.0.1:51564)
0619 20:37:12.010749 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:12.010758 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:12.010800 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:12.010970 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:12.010971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:12.010972 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:37:12.011067 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:12.011069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.011965 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.011966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:12.013349 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:12.013351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.014201 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.014203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:12.014255 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:12.014259 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:37:12.014310 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 20:37:12.014348 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":38,"thread_start_us":61,"threads_started":1}
I20260619 20:37:12.014947 4093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:12.010282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35584 (local address 127.25.62.131:46739)
0619 20:37:12.010634 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:12.010638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:12.010647 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:37:12.010682 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:12.010683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:12.010697 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:12.010775 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:37:12.011123 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.012181 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.013251 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.013951 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.014020 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:12.014079 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:12.014801 (+ 722us) server_negotiation.cc:299] Negotiation successful
0619 20:37:12.014845 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":74,"threads_started":1}
I20260619 20:37:12.015376 3883 tablet_service.cc:1467] Processing CreateTablet for tablet 650d43ab04de44fd8e2d2b0973ff4a92 (DEFAULT_TABLE table=test-table [id=1ccf8c7118394602b57759db23a17a89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:37:12.015530 4017 tablet_service.cc:1467] Processing CreateTablet for tablet 650d43ab04de44fd8e2d2b0973ff4a92 (DEFAULT_TABLE table=test-table [id=1ccf8c7118394602b57759db23a17a89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:37:12.015530 3749 tablet_service.cc:1467] Processing CreateTablet for tablet 650d43ab04de44fd8e2d2b0973ff4a92 (DEFAULT_TABLE table=test-table [id=1ccf8c7118394602b57759db23a17a89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:37:12.015673 3883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 650d43ab04de44fd8e2d2b0973ff4a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:37:12.015836 3749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 650d43ab04de44fd8e2d2b0973ff4a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:37:12.015836 4017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 650d43ab04de44fd8e2d2b0973ff4a92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:37:12.019398 4099 tablet_bootstrap.cc:492] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8: Bootstrap starting.
I20260619 20:37:12.019371 4098 tablet_bootstrap.cc:492] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56: Bootstrap starting.
I20260619 20:37:12.019606 4097 tablet_bootstrap.cc:492] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508: Bootstrap starting.
I20260619 20:37:12.020100 4099 tablet_bootstrap.cc:654] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8: Neither blocks nor log segments found. Creating new log.
I20260619 20:37:12.020102 4098 tablet_bootstrap.cc:654] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56: Neither blocks nor log segments found. Creating new log.
I20260619 20:37:12.020304 4097 tablet_bootstrap.cc:654] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508: Neither blocks nor log segments found. Creating new log.
I20260619 20:37:12.020730 4099 log.cc:824] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8: Log is configured to *not* fsync() on all Append() calls
I20260619 20:37:12.020763 4098 log.cc:824] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56: Log is configured to *not* fsync() on all Append() calls
I20260619 20:37:12.020874 4097 log.cc:824] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508: Log is configured to *not* fsync() on all Append() calls
I20260619 20:37:12.021548 4099 tablet_bootstrap.cc:492] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8: No bootstrap required, opened a new log
I20260619 20:37:12.021541 4098 tablet_bootstrap.cc:492] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56: No bootstrap required, opened a new log
I20260619 20:37:12.021616 4099 ts_tablet_manager.cc:1397] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:37:12.021616 4098 ts_tablet_manager.cc:1397] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:37:12.021695 4097 tablet_bootstrap.cc:492] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508: No bootstrap required, opened a new log
I20260619 20:37:12.021765 4097 ts_tablet_manager.cc:1397] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 20:37:12.023115 4097 raft_consensus.cc:348] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd0cd8d53a40dea61c996bfa488508" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33013 } } peers { permanent_uuid: "e387c43bc0314bd79a5feef8377915e8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 46739 } } peers { permanent_uuid: "f932d7fd27e645f396cd1fa339170d56" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34455 } }
I20260619 20:37:12.023284 4097 raft_consensus.cc:374] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:37:12.023334 4097 raft_consensus.cc:729] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40fd0cd8d53a40dea61c996bfa488508, State: Initialized, Role: FOLLOWER
I20260619 20:37:12.023447 4097 consensus_queue.cc:260] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd0cd8d53a40dea61c996bfa488508" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33013 } } peers { permanent_uuid: "e387c43bc0314bd79a5feef8377915e8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 46739 } } peers { permanent_uuid: "f932d7fd27e645f396cd1fa339170d56" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34455 } }
I20260619 20:37:12.023535 4098 raft_consensus.cc:348] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd0cd8d53a40dea61c996bfa488508" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33013 } } peers { permanent_uuid: "e387c43bc0314bd79a5feef8377915e8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 46739 } } peers { permanent_uuid: "f932d7fd27e645f396cd1fa339170d56" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34455 } }
I20260619 20:37:12.023535 4099 raft_consensus.cc:348] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd0cd8d53a40dea61c996bfa488508" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33013 } } peers { permanent_uuid: "e387c43bc0314bd79a5feef8377915e8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 46739 } } peers { permanent_uuid: "f932d7fd27e645f396cd1fa339170d56" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34455 } }
I20260619 20:37:12.023694 4099 raft_consensus.cc:374] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:37:12.023694 4098 raft_consensus.cc:374] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:37:12.023736 4099 raft_consensus.cc:729] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e387c43bc0314bd79a5feef8377915e8, State: Initialized, Role: FOLLOWER
I20260619 20:37:12.023736 4098 raft_consensus.cc:729] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f932d7fd27e645f396cd1fa339170d56, State: Initialized, Role: FOLLOWER
I20260619 20:37:12.023742 4097 ts_tablet_manager.cc:1428] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 20:37:12.023841 4099 consensus_queue.cc:260] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd0cd8d53a40dea61c996bfa488508" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33013 } } peers { permanent_uuid: "e387c43bc0314bd79a5feef8377915e8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 46739 } } peers { permanent_uuid: "f932d7fd27e645f396cd1fa339170d56" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34455 } }
I20260619 20:37:12.023855 4098 consensus_queue.cc:260] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd0cd8d53a40dea61c996bfa488508" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33013 } } peers { permanent_uuid: "e387c43bc0314bd79a5feef8377915e8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 46739 } } peers { permanent_uuid: "f932d7fd27e645f396cd1fa339170d56" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34455 } }
I20260619 20:37:12.024108 4099 ts_tablet_manager.cc:1428] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 20:37:12.024108 4098 ts_tablet_manager.cc:1428] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 20:37:12.024152 3815 heartbeater.cc:503] Master 127.25.62.190:44673 was elected leader, sending a full tablet report...
I20260619 20:37:12.024257 4083 heartbeater.cc:503] Master 127.25.62.190:44673 was elected leader, sending a full tablet report...
I20260619 20:37:12.024250 3949 heartbeater.cc:503] Master 127.25.62.190:44673 was elected leader, sending a full tablet report...
I20260619 20:37:12.033165 4105 raft_consensus.cc:482] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:37:12.033241 4105 raft_consensus.cc:504] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd0cd8d53a40dea61c996bfa488508" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33013 } } peers { permanent_uuid: "e387c43bc0314bd79a5feef8377915e8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 46739 } } peers { permanent_uuid: "f932d7fd27e645f396cd1fa339170d56" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34455 } }
I20260619 20:37:12.033593 4105 leader_election.cc:288] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40fd0cd8d53a40dea61c996bfa488508 (127.25.62.130:33013), e387c43bc0314bd79a5feef8377915e8 (127.25.62.131:46739)
I20260619 20:37:12.036504 4093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:12.033859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:39085 (local address 127.25.62.131:46739)
0619 20:37:12.033896 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:12.033898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:12.034073 (+ 175us) server_negotiation.cc:407] Connection header received
0619 20:37:12.034117 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:12.034118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:12.034128 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:12.034183 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:37:12.034309 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.034800 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.035799 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.036164 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.036246 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:12.036254 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:12.036414 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 20:37:12.036447 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:37:12.036520 4106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:12.033760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:46739 (local address 127.25.62.129:39085)
0619 20:37:12.033976 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:12.033984 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:12.034057 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:12.034192 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:12.034192 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:12.034194 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:37:12.034249 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:12.034250 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.034886 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.034887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:12.035697 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:12.035699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.036245 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.036246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:12.036319 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:12.036328 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 20:37:12.036400 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 20:37:12.036439 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":84,"thread_start_us":88,"threads_started":1}
I20260619 20:37:12.036590 4094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:12.033870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:49269 (local address 127.25.62.130:33013)
0619 20:37:12.033915 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:12.033918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:12.033926 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:37:12.033953 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:12.033953 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:12.033965 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:12.034031 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:37:12.034254 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.034792 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.035816 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.036206 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.036306 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:12.036314 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:12.036484 (+ 170us) server_negotiation.cc:299] Negotiation successful
0619 20:37:12.036519 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:37:12.036801 3817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:12.033760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:33013 (local address 127.25.62.129:49269)
0619 20:37:12.033819 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:12.033833 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:12.033882 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:12.034046 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:12.034047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:12.034049 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:37:12.034118 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:12.034120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.034889 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.034890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:12.035699 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:12.035702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.036306 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.036307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:12.036393 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:12.036396 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:37:12.036465 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 20:37:12.036505 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":28}
I20260619 20:37:12.037139 4037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "650d43ab04de44fd8e2d2b0973ff4a92" candidate_uuid: "f932d7fd27e645f396cd1fa339170d56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e387c43bc0314bd79a5feef8377915e8" is_pre_election: true
I20260619 20:37:12.037156 3903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "650d43ab04de44fd8e2d2b0973ff4a92" candidate_uuid: "f932d7fd27e645f396cd1fa339170d56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40fd0cd8d53a40dea61c996bfa488508" is_pre_election: true
I20260619 20:37:12.037264 4037 raft_consensus.cc:2454] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f932d7fd27e645f396cd1fa339170d56 in term 0.
I20260619 20:37:12.037292 3903 raft_consensus.cc:2454] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f932d7fd27e645f396cd1fa339170d56 in term 0.
I20260619 20:37:12.037497 3705 leader_election.cc:302] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e387c43bc0314bd79a5feef8377915e8, f932d7fd27e645f396cd1fa339170d56; no voters:
I20260619 20:37:12.037628 4105 raft_consensus.cc:2781] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:37:12.037683 4105 raft_consensus.cc:482] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:37:12.037705 4105 raft_consensus.cc:3037] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:37:12.038327 4105 raft_consensus.cc:504] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd0cd8d53a40dea61c996bfa488508" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33013 } } peers { permanent_uuid: "e387c43bc0314bd79a5feef8377915e8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 46739 } } peers { permanent_uuid: "f932d7fd27e645f396cd1fa339170d56" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34455 } }
I20260619 20:37:12.038457 4105 leader_election.cc:288] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [CANDIDATE]: Term 1 election: Requested vote from peers 40fd0cd8d53a40dea61c996bfa488508 (127.25.62.130:33013), e387c43bc0314bd79a5feef8377915e8 (127.25.62.131:46739)
I20260619 20:37:12.038596 3903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "650d43ab04de44fd8e2d2b0973ff4a92" candidate_uuid: "f932d7fd27e645f396cd1fa339170d56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40fd0cd8d53a40dea61c996bfa488508"
I20260619 20:37:12.038609 4037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "650d43ab04de44fd8e2d2b0973ff4a92" candidate_uuid: "f932d7fd27e645f396cd1fa339170d56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e387c43bc0314bd79a5feef8377915e8"
I20260619 20:37:12.038689 4037 raft_consensus.cc:3037] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:37:12.038695 3903 raft_consensus.cc:3037] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:37:12.039330 4037 raft_consensus.cc:2454] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f932d7fd27e645f396cd1fa339170d56 in term 1.
I20260619 20:37:12.039481 3705 leader_election.cc:302] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e387c43bc0314bd79a5feef8377915e8, f932d7fd27e645f396cd1fa339170d56; no voters:
I20260619 20:37:12.039548 3903 raft_consensus.cc:2454] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f932d7fd27e645f396cd1fa339170d56 in term 1.
I20260619 20:37:12.039572 4105 raft_consensus.cc:2781] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:37:12.039716 4105 raft_consensus.cc:686] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [term 1 LEADER]: Becoming Leader. State: Replica: f932d7fd27e645f396cd1fa339170d56, State: Running, Role: LEADER
I20260619 20:37:12.039803 4105 consensus_queue.cc:237] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd0cd8d53a40dea61c996bfa488508" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33013 } } peers { permanent_uuid: "e387c43bc0314bd79a5feef8377915e8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 46739 } } peers { permanent_uuid: "f932d7fd27e645f396cd1fa339170d56" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34455 } }
I20260619 20:37:12.040556 3621 catalog_manager.cc:5495] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 reported cstate change: term changed from 0 to 1, leader changed from <none> to f932d7fd27e645f396cd1fa339170d56 (127.25.62.129). New cstate: current_term: 1 leader_uuid: "f932d7fd27e645f396cd1fa339170d56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd0cd8d53a40dea61c996bfa488508" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e387c43bc0314bd79a5feef8377915e8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 46739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f932d7fd27e645f396cd1fa339170d56" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34455 } health_report { overall_health: HEALTHY } } }
I20260619 20:37:12.052070 4096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:12.050041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51578 (local address 127.25.62.129:34455)
0619 20:37:12.050098 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:12.050100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:12.050140 (+ 40us) server_negotiation.cc:407] Connection header received
0619 20:37:12.050168 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:12.050169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:12.050185 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:12.050235 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:37:12.050338 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.050782 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.051463 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:12.051641 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:12.051690 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:12.051811 (+ 121us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:37:12.051897 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:37:12.051955 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:12.051986 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 20:37:12.052002 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 20:37:12.053704 4037 raft_consensus.cc:1261] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8 [term 1 FOLLOWER]: Refusing update from remote peer f932d7fd27e645f396cd1fa339170d56: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:37:12.053725 3903 raft_consensus.cc:1261] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508 [term 1 FOLLOWER]: Refusing update from remote peer f932d7fd27e645f396cd1fa339170d56: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:37:12.054045 4107 consensus_queue.cc:1035] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40fd0cd8d53a40dea61c996bfa488508" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:37:12.054171 4105 consensus_queue.cc:1035] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e387c43bc0314bd79a5feef8377915e8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 46739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:37:12.056859 4112 mvcc.cc:205] Tried to move back new op lower bound from 7298668265689653248 to 7298668265635557376. Current Snapshot: MvccSnapshot[applied={T|T < 7298668265689653248}]
I20260619 20:37:12.058524 3622 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:37:12.058614 3622 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:37:12.059370 3622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 877bc2df5ce740b2909708a6f9442439: Sending DeleteTablet for 3 replicas of tablet 650d43ab04de44fd8e2d2b0973ff4a92
I20260619 20:37:12.059682 4017 tablet_service.cc:1514] Processing DeleteTablet for tablet 650d43ab04de44fd8e2d2b0973ff4a92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:37:12 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:35584
I20260619 20:37:12.059706 3883 tablet_service.cc:1514] Processing DeleteTablet for tablet 650d43ab04de44fd8e2d2b0973ff4a92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:37:12 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:47926
I20260619 20:37:12.059968 3749 tablet_service.cc:1514] Processing DeleteTablet for tablet 650d43ab04de44fd8e2d2b0973ff4a92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:37:12 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:51564
I20260619 20:37:12.060187 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 3686
I20260619 20:37:12.060310 4121 tablet_replica.cc:330] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56: stopping tablet replica
I20260619 20:37:12.060420 4121 raft_consensus.cc:2229] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:37:12.060563 4121 raft_consensus.cc:2258] T 650d43ab04de44fd8e2d2b0973ff4a92 P f932d7fd27e645f396cd1fa339170d56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:37:12.060842 4119 tablet_replica.cc:330] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508: stopping tablet replica
I20260619 20:37:12.060952 4119 raft_consensus.cc:2229] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:37:12.061074 4119 raft_consensus.cc:2258] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:37:12.059908 4120 tablet_replica.cc:330] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8: stopping tablet replica
I20260619 20:37:12.061234 4120 raft_consensus.cc:2229] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:37:12.061321 4120 raft_consensus.cc:2258] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:37:12.061728 4119 ts_tablet_manager.cc:1905] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:37:12.062206 4120 ts_tablet_manager.cc:1905] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:37:12.063463 4119 ts_tablet_manager.cc:1918] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:37:12.063530 4120 ts_tablet_manager.cc:1918] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:37:12.063552 4119 log.cc:1196] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/650d43ab04de44fd8e2d2b0973ff4a92
I20260619 20:37:12.063584 4120 log.cc:1196] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/650d43ab04de44fd8e2d2b0973ff4a92
I20260619 20:37:12.063854 4120 ts_tablet_manager.cc:1939] T 650d43ab04de44fd8e2d2b0973ff4a92 P e387c43bc0314bd79a5feef8377915e8: Deleting consensus metadata
I20260619 20:37:12.063855 4119 ts_tablet_manager.cc:1939] T 650d43ab04de44fd8e2d2b0973ff4a92 P 40fd0cd8d53a40dea61c996bfa488508: Deleting consensus metadata
I20260619 20:37:12.064211 3610 catalog_manager.cc:4841] TS 40fd0cd8d53a40dea61c996bfa488508 (127.25.62.130:33013): tablet 650d43ab04de44fd8e2d2b0973ff4a92 (table test-table [id=1ccf8c7118394602b57759db23a17a89]) successfully deleted
I20260619 20:37:12.064251 3611 catalog_manager.cc:4841] TS e387c43bc0314bd79a5feef8377915e8 (127.25.62.131:46739): tablet 650d43ab04de44fd8e2d2b0973ff4a92 (table test-table [id=1ccf8c7118394602b57759db23a17a89]) successfully deleted
W20260619 20:37:12.066532 3611 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.62.129:34455 (error 108)
I20260619 20:37:12.066716 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 3820
I20260619 20:37:12.067298 4091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:12.066854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:34455 (local address 127.0.0.1:51586)
0619 20:37:12.067227 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:12.067261 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.129:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355}
W20260619 20:37:12.067425 3611 catalog_manager.cc:4584] TS f932d7fd27e645f396cd1fa339170d56 (127.25.62.129:34455): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 650d43ab04de44fd8e2d2b0973ff4a92: Network error: Client connection negotiation failed: client connection to 127.25.62.129:34455: connect: Connection refused (error 111)
I20260619 20:37:12.071794 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 3954
I20260619 20:37:12.076222 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 3592
2026-06-19T20:37:12Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1854 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:37:12 dist-test-slave-0t40 krb5kdc[4125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:37:12 dist-test-slave-0t40 krb5kdc[4125](info): set up 2 sockets
Jun 19 20:37:12 dist-test-slave-0t40 krb5kdc[4125](info): commencing operation
krb5kdc: starting...
W20260619 20:37:13.118784 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:37:13 dist-test-slave-0t40 krb5kdc[4125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:37:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:37:13Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.25.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.25.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.25.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/rangeradmin_127.25.62.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.25.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/rangeradmin_127.25.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.25.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.25.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.25.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/rangerlookup_127.25.62.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.25.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/rangerlookup_127.25.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.25.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/HTTP_127.25.62.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/HTTP_127.25.62.148@KRBTEST.COM.keytab.
I20260619 20:37:13.277799 25850 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskXT8PDl/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/postgres -l logfile start
2026-06-19 20:37:15.766 UTC [4155] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 20:37:15.766 UTC [4155] LOG: listening on IPv4 address "127.25.62.148", port 54899
2026-06-19 20:37:15.769 UTC [4155] LOG: listening on Unix socket "/tmp/.s.PGSQL.54899"
2026-06-19 20:37:15.790 UTC [4158] LOG: database system was shut down at 2026-06-19 20:37:14 UTC
2026-06-19 20:37:15.793 UTC [4155] LOG: database system is ready to accept connections
I20260619 20:37:16.762598 25850 mini_postgres.cc:96] Postgres bound to 54899
2026-06-19 20:37:16.768 UTC [4166] FATAL: database "slave" does not exist
127.25.62.148:54899 - accepting connections
I20260619 20:37:16.769452 25850 mini_ranger.cc:161] Starting Ranger...
I20260619 20:37:16.783692 25850 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 20:37:17.273967 25850 mini_ranger.cc:87] Created ranger Postgres database
I20260619 20:37:17.274060 25850 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ranger-admin
2026-06-19 20:37:17,544 [I] DB FLAVOR :POSTGRES
2026-06-19 20:37:17,545 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 20:37:17,545 [I] Checking connection..
2026-06-19 20:37:17,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 20:37:17,798 [I] Checking connection passed.
2026-06-19 20:37:17,798 [I] --------- Verifying version history table ---------
2026-06-19 20:37:17,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 20:37:18,058 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 20:37:18,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 20:37:18,285 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 20:37:18,286 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 20:37:18,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 20:37:18.541 UTC [4230] WARNING: there is no transaction in progress
2026-06-19 20:37:18,547 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 20:37:18,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 20:37:18,793 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 20:37:18,794 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 20:37:18,794 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 20:37:19,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 20:37:19,287 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-0t40.c.gcp-upstream.internal','N') ;"
2026-06-19 20:37:19,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 20:37:19,777 [I] Table x_portal_user does not exist in database ranger
2026-06-19 20:37:19,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 20:37:20,027 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 20:37:20,028 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 20:37:20,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 20:37:20,521 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 20:37:20,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 20:37:21.490 UTC [4338] WARNING: there is no transaction in progress
2026-06-19 20:37:21.506 UTC [4338] WARNING: there is no transaction in progress
2026-06-19 20:37:21.523 UTC [4338] WARNING: there is no transaction in progress
2026-06-19 20:37:21.537 UTC [4338] WARNING: there is no transaction in progress
2026-06-19 20:37:21.553 UTC [4338] WARNING: there is no transaction in progress
2026-06-19 20:37:21.572 UTC [4338] WARNING: there is no transaction in progress
2026-06-19 20:37:21.631 UTC [4338] WARNING: there is no transaction in progress
2026-06-19 20:37:21.638 UTC [4338] WARNING: there is no transaction in progress
2026-06-19 20:37:21.652 UTC [4338] WARNING: there is no transaction in progress
2026-06-19 20:37:21.662 UTC [4338] WARNING: there is no transaction in progress
2026-06-19 20:37:21.677 UTC [4338] WARNING: there is no transaction in progress
2026-06-19 20:37:21.693 UTC [4338] WARNING: there is no transaction in progress
2026-06-19 20:37:21.704 UTC [4338] WARNING: there is no transaction in progress
2026-06-19 20:37:21.711 UTC [4338] WARNING: there is no transaction in progress
2026-06-19 20:37:21.718 UTC [4338] WARNING: there is no transaction in progress
2026-06-19 20:37:21.725 UTC [4338] WARNING: there is no transaction in progress
2026-06-19 20:37:22,350 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 20:37:22,351 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 20:37:22,594 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 20:37:22,595 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 20:37:22,823 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 20:37:22,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 20:37:23,077 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 20:37:23,077 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 20:37:23,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 20:37:23,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-0t40.c.gcp-upstream.internal';"
2026-06-19 20:37:23,784 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 20:37:23,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 20:37:24,023 [I] DB_PATCHES have already been applied
I20260619 20:37:24.028735 25850 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ranger-admin:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskXT8PDl/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 20:37:24.028805 25850 mini_ranger.cc:193] Using host: 127.25.62.148
I20260619 20:37:24.031910 25850 mini_ranger.cc:227] Ranger admin URL: http://127.25.62.148:42249
Jun 19, 2026 8:37:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:37:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:37:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 8:37:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 8:37:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 8:37:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 8:37:25 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 20:37:26.227923 25850 mini_ranger.cc:160] Time spent starting Ranger: real 9.458s user 0.000s sys 0.010s
Jun 19, 2026 8:37:44 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 8:37:44 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 8:37:44 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 8:37:44 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 8:37:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 8:37:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 8:37:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 8:37:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 8:37:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 8:37:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 8:37:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 8:37:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 8:37:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 8:37:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 8:37:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 8:37:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 8:37:44 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 8:37:45 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 20:37:46.349346 25850 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.25.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.25.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.25.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/rangerkms_127.25.62.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.25.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/rangerkms_127.25.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.62.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.25.62.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.62.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/HTTP_127.25.62.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.62.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/HTTP_127.25.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 20:37:46 dist-test-slave-0t40 krb5kdc[4125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901466, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 20:37:46.437336 25850 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 20:37:46.446486 25850 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 20:37:47.327548 25850 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 20:37:47.327620 25850 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ranger-kms
I20260619 20:37:47.327636 25850 mini_ranger_kms.cc:227] Using postgres at 127.25.62.148:54899
2026-06-19 20:37:47,512 [I] DB FLAVOR :POSTGRES
2026-06-19 20:37:47,512 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 20:37:47,512 [I] Checking connection
2026-06-19 20:37:47,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 20:37:47,749 [I] connection success
2026-06-19 20:37:47,749 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 20:37:47,750 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 20:37:47,750 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 20:37:48,005 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 20:37:48,005 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 20:37:48,006 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 20:37:48,006 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:54899/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 20:37:48,280 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 20:37:48.767863 25850 mini_ranger_kms.cc:310] Created kms service
I20260619 20:37:48.962617 25850 mini_ranger_kms.cc:326] Created kudu user
I20260619 20:37:49.007688 25850 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 20:37:49 dist-test-slave-0t40 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901466, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
I20260619 20:37:49.387038 25850 mini_ranger_kms.cc:384] Added ranger policy
I20260619 20:37:49.387156 25850 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ranger-kms:/tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 20:37:49.387177 25850 mini_ranger_kms.cc:240] Using host: 127.25.62.148
I20260619 20:37:49.390551 25850 mini_ranger_kms.cc:276] Ranger KMS PID: 4519
I20260619 20:37:49.390677 25850 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.25.62.148:43443
Jun 19, 2026 8:37:50 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:37:50 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:37:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 8:37:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 8:37:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 8:37:50 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 20:37:51.646281 25850 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.209s user 0.000s sys 0.168s
I20260619 20:37:51.646391 25850 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 20:37:51.646466 25850 mini_ranger_kms.cc:401] 127.25.62.148:43443/kms/v1/keys
Jun 19 20:37:51 dist-test-slave-0t40 krb5kdc[4125](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781901471, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.62.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 20:37:52 dist-test-slave-0t40 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901471, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.62.148@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
Jun 19 20:37:52 dist-test-slave-0t40 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901471, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.62.148@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
Jun 19 20:37:54 dist-test-slave-0t40 krb5kdc[4125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:37:54.751233 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:41199
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:42827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:41199
--ranger_config_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.62.148:43443/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:37:54.832366 4570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:37:54.832577 4570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:37:54.832599 4570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:37:54.833745 4570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:37:54.833786 4570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:37:54.833798 4570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:37:54.833812 4570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:37:54.833824 4570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:37:54.835381 4570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:42827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/master-0/wal
--ranger_kms_url=127.25.62.148:43443/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:41199
--ranger_config_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:41199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:37:54.835613 4570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:37:54.835837 4570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:37:54.840869 4570 server_base.cc:1034] running on GCE node
W20260619 20:37:54.840911 4576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:54.841107 4578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:54.841198 4575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:54.841473 4570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:37:54.841836 4570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:37:54.843014 4570 hybrid_clock.cc:648] HybridClock initialized: now 1781901474842988 us; error 40 us; skew 500 ppm
Jun 19 20:37:54 dist-test-slave-0t40 krb5kdc[4125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901474, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:37:54.850175 4570 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:37:54.850687 4570 webserver.cc:460] Webserver started at http://127.25.62.190:43101/ using document root <none> and password file <none>
I20260619 20:37:54.850929 4570 fs_manager.cc:359] Metadata directory not provided
I20260619 20:37:54.850970 4570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:37:54.851100 4570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:37:54 dist-test-slave-0t40 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901474, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
I20260619 20:37:54.908946 4570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "101fc7bfc6744d62be14c16ccbc1be5f"
format_stamp: "Formatted at 2026-06-19 20:37:54 on dist-test-slave-0t40"
server_key: "7bca23fc32d9a6adf85e876d6d8f0d70"
server_key_iv: "041ea32f6a82deba0d8e6c5218ae2994"
server_key_version: "kuduclusterkey@0"
I20260619 20:37:54.909500 4570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "101fc7bfc6744d62be14c16ccbc1be5f"
format_stamp: "Formatted at 2026-06-19 20:37:54 on dist-test-slave-0t40"
server_key: "7bca23fc32d9a6adf85e876d6d8f0d70"
server_key_iv: "041ea32f6a82deba0d8e6c5218ae2994"
server_key_version: "kuduclusterkey@0"
I20260619 20:37:54.910912 4570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:37:54.925274 4585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:54.925458 4570 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:37:54.925554 4570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "101fc7bfc6744d62be14c16ccbc1be5f"
format_stamp: "Formatted at 2026-06-19 20:37:54 on dist-test-slave-0t40"
server_key: "7bca23fc32d9a6adf85e876d6d8f0d70"
server_key_iv: "041ea32f6a82deba0d8e6c5218ae2994"
server_key_version: "kuduclusterkey@0"
I20260619 20:37:54.925657 4570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:37:54.940522 4570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:37:54.942592 4570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:37:54.942751 4570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:37:54.947875 4570 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:41199
I20260619 20:37:54.947896 4637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:41199 every 8 connection(s)
I20260619 20:37:54.948240 4570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:37:54.948909 4638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:37:54.951130 4638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f: Bootstrap starting.
I20260619 20:37:54.951809 4638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f: Neither blocks nor log segments found. Creating new log.
I20260619 20:37:54.952502 4638 log.cc:824] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f: Log is configured to *not* fsync() on all Append() calls
I20260619 20:37:54.953208 4638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f: No bootstrap required, opened a new log
I20260619 20:37:54.955205 4638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "101fc7bfc6744d62be14c16ccbc1be5f" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41199 } }
I20260619 20:37:54.955345 4638 raft_consensus.cc:374] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:37:54.955369 4638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101fc7bfc6744d62be14c16ccbc1be5f, State: Initialized, Role: FOLLOWER
I20260619 20:37:54.955525 4638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "101fc7bfc6744d62be14c16ccbc1be5f" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41199 } }
I20260619 20:37:54.955592 4638 raft_consensus.cc:388] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:37:54.955631 4638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:37:54.955675 4638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:37:54.956180 4638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "101fc7bfc6744d62be14c16ccbc1be5f" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41199 } }
I20260619 20:37:54.956261 4638 leader_election.cc:302] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 101fc7bfc6744d62be14c16ccbc1be5f; no voters:
I20260619 20:37:54.956457 4638 leader_election.cc:288] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:37:54.956482 4641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:37:54.956611 4641 raft_consensus.cc:686] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f [term 1 LEADER]: Becoming Leader. State: Replica: 101fc7bfc6744d62be14c16ccbc1be5f, State: Running, Role: LEADER
I20260619 20:37:54.956712 4638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:37:54.956758 4641 consensus_queue.cc:237] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "101fc7bfc6744d62be14c16ccbc1be5f" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41199 } }
I20260619 20:37:54.956983 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 4570
I20260619 20:37:54.957114 4641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 101fc7bfc6744d62be14c16ccbc1be5f. Latest consensus state: current_term: 1 leader_uuid: "101fc7bfc6744d62be14c16ccbc1be5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "101fc7bfc6744d62be14c16ccbc1be5f" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41199 } } }
I20260619 20:37:54.957168 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:37:54.957192 4641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f [sys.catalog]: This master's current role is: LEADER
I20260619 20:37:54.957365 25850 external_mini_cluster.cc:1428] Setting key 51e009d618f38c87d274ad4747a5275a
I20260619 20:37:54.958146 4643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "101fc7bfc6744d62be14c16ccbc1be5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "101fc7bfc6744d62be14c16ccbc1be5f" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41199 } } }
I20260619 20:37:54.958281 4643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f [sys.catalog]: This master's current role is: LEADER
I20260619 20:37:54.958278 4638 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 20:37:54.958562 4647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:37:54.961421 4647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:37:54.968254 4647 catalog_manager.cc:1347] Generated new cluster ID: ecd8eac2970448c0989c42c954ae6193
I20260619 20:37:54.968312 4647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:37:54 dist-test-slave-0t40 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:37:54.973521 4654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:54.957796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43186 (local address 127.25.62.190:41199)
0619 20:37:54.960994 (+ 3198us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:54.961020 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:54.961107 (+ 87us) server_negotiation.cc:407] Connection header received
0619 20:37:54.961444 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:54.961459 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:54.961553 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:54.961741 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:54.962439 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:54.964377 (+ 1938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:54.965911 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:54.966610 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:54.966735 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:54.969638 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:37:54.969642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:37:54.969642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:37:54.969654 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:37:54.971689 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:54.972052 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:54.972055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:54.972055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:54.972139 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:54.972296 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:54.972297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:54.972297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:54.972582 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:37:54.972721 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:54.973145 (+ 424us) server_negotiation.cc:299] Negotiation successful
0619 20:37:54.973261 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2945,"thread_start_us":110,"threads_started":1}
I20260619 20:37:55.009255 4647 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:37:55.010185 4647 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:37:55.015657 4647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f: Generated new TSK 0
I20260619 20:37:55.016088 4647 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
I20260619 20:37:55.703929 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.62.148:43443/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41199
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:42827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:37:55.811234 4683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:37:55.811555 4683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:37:55.811599 4683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:37:55.813608 4683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:37:55.813685 4683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:37:55.813735 4683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:37:55.816162 4683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:42827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.25.62.148:43443/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:37:55.816618 4683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:37:55.816941 4683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:37:55.823659 4689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:55.824110 4688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:55.824172 4691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:55.825460 4683 server_base.cc:1034] running on GCE node
I20260619 20:37:55.825719 4683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:37:55.825979 4683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:37:55.827190 4683 hybrid_clock.cc:648] HybridClock initialized: now 1781901475827163 us; error 43 us; skew 500 ppm
Jun 19 20:37:55 dist-test-slave-0t40 krb5kdc[4125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901475, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:37:55.830039 4683 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:37:55.830597 4683 webserver.cc:460] Webserver started at http://127.25.62.129:38897/ using document root <none> and password file <none>
I20260619 20:37:55.830942 4683 fs_manager.cc:359] Metadata directory not provided
I20260619 20:37:55.831063 4683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:37:55.831246 4683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:37:55 dist-test-slave-0t40 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901475, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
I20260619 20:37:55.853969 4683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "a0e6a88c02ba43ceb2a65f828e78d49f"
format_stamp: "Formatted at 2026-06-19 20:37:55 on dist-test-slave-0t40"
server_key: "a708cb827b667701fb4806413be54946"
server_key_iv: "6e5aeff3fe033e9caa82a0072b2f8dc4"
server_key_version: "kuduclusterkey@0"
I20260619 20:37:55.854988 4683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "a0e6a88c02ba43ceb2a65f828e78d49f"
format_stamp: "Formatted at 2026-06-19 20:37:55 on dist-test-slave-0t40"
server_key: "a708cb827b667701fb4806413be54946"
server_key_iv: "6e5aeff3fe033e9caa82a0072b2f8dc4"
server_key_version: "kuduclusterkey@0"
I20260619 20:37:55.856910 4683 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260619 20:37:55.875787 4700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:55.876053 4683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:37:55.876149 4683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "a0e6a88c02ba43ceb2a65f828e78d49f"
format_stamp: "Formatted at 2026-06-19 20:37:55 on dist-test-slave-0t40"
server_key: "a708cb827b667701fb4806413be54946"
server_key_iv: "6e5aeff3fe033e9caa82a0072b2f8dc4"
server_key_version: "kuduclusterkey@0"
I20260619 20:37:55.876242 4683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:37:55.889490 4683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:37:55.891978 4683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:37:55.892175 4683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:37:55.892441 4683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:37:55.892856 4683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:37:55.892961 4683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:55.893092 4683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:37:55.893182 4683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:55.900326 4683 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:33443
I20260619 20:37:55.901006 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 4683
I20260619 20:37:55.901124 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:37:55.901340 25850 external_mini_cluster.cc:1428] Setting key 8d22e1a8514c5d2bd1622c6b11cf636c
I20260619 20:37:55.900817 4683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:37:55.910346 4813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:33443 every 8 connection(s)
Jun 19 20:37:55 dist-test-slave-0t40 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901475, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.130@KRBTEST.COM" created.
I20260619 20:37:55.924427 4818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:55.911093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:41199 (local address 127.25.62.129:36633)
0619 20:37:55.911770 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:55.911820 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:55.912469 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:55.912838 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:55.912844 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:55.913151 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:55.913438 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:55.913446 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:55.914485 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:55.914486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:55.915744 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:55.915747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:55.916293 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:55.916294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:55.916437 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:55.917013 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:55.917024 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:55.918683 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:37:55.920714 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:37:55.920721 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:37:55.920724 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:37:55.921032 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:37:55.921399 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:37:55.921400 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:37:55.921400 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:37:55.921535 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:37:55.923710 (+ 2175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:37:55.923717 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:37:55.923892 (+ 175us) client_negotiation.cc:769] Sending connection context
0619 20:37:55.924004 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 20:37:55.924170 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":90,"threads_started":1}
I20260619 20:37:55.924978 4817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:55.911099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:36633 (local address 127.25.62.190:41199)
0619 20:37:55.912159 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:55.912162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:55.912174 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:37:55.912590 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:55.912592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:55.912605 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:55.912684 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:55.913556 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:55.914377 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:55.915843 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:55.916156 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:55.916225 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:55.918788 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:37:55.918789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:37:55.918790 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:37:55.918798 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:37:55.920627 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:55.921149 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:55.921151 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:55.921151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:55.921314 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:55.921607 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:55.921608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:55.921608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:55.921973 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:37:55.922044 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:55.924792 (+ 2748us) server_negotiation.cc:299] Negotiation successful
0619 20:37:55.924881 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1033,"thread_start_us":79,"threads_started":1}
I20260619 20:37:55.925860 4815 heartbeater.cc:347] Connected to a master server at 127.25.62.190:41199
I20260619 20:37:55.926002 4815 heartbeater.cc:464] Registering TS with master...
I20260619 20:37:55.926242 4815 heartbeater.cc:511] Master 127.25.62.190:41199 requested a full tablet report, sending...
I20260619 20:37:55.926801 4602 ts_manager.cc:194] Registered new tserver with Master: a0e6a88c02ba43ceb2a65f828e78d49f (127.25.62.129:33443)
I20260619 20:37:55.927711 4602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:36633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
I20260619 20:37:55.958559 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.62.148:43443/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41199
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:42827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 20:37:56 dist-test-slave-0t40 krb5kdc[4125](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781901476, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 20:37:56.077410 4820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:37:56.077746 4820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:37:56.077829 4820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:37:56.079545 4820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:37:56.079746 4820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:37:56.079867 4820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:37:56.081979 4820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:42827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.25.62.148:43443/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:37:56.082415 4820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:37:56.082695 4820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:37:56.088728 4828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:56.089104 4827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:56.089295 4830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:56.090312 4820 server_base.cc:1034] running on GCE node
I20260619 20:37:56.090533 4820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:37:56.090787 4820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:37:56.094067 4820 hybrid_clock.cc:648] HybridClock initialized: now 1781901476094038 us; error 46 us; skew 500 ppm
Jun 19 20:37:56 dist-test-slave-0t40 krb5kdc[4125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901476, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:37:56.097289 4820 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:37:56.097779 4820 webserver.cc:460] Webserver started at http://127.25.62.130:43323/ using document root <none> and password file <none>
I20260619 20:37:56.098007 4820 fs_manager.cc:359] Metadata directory not provided
I20260619 20:37:56.098054 4820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:37:56.098172 4820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:37:56 dist-test-slave-0t40 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901476, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
I20260619 20:37:56.112661 4820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "c4ff6ec8fdf4439ca3d8966917fe1983"
format_stamp: "Formatted at 2026-06-19 20:37:56 on dist-test-slave-0t40"
server_key: "d2502d79c2cbde4a42219bafc9172bca"
server_key_iv: "9b7264b82308b7d7539bfa40dd503667"
server_key_version: "kuduclusterkey@0"
I20260619 20:37:56.113138 4820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "c4ff6ec8fdf4439ca3d8966917fe1983"
format_stamp: "Formatted at 2026-06-19 20:37:56 on dist-test-slave-0t40"
server_key: "d2502d79c2cbde4a42219bafc9172bca"
server_key_iv: "9b7264b82308b7d7539bfa40dd503667"
server_key_version: "kuduclusterkey@0"
I20260619 20:37:56.116433 4820 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260619 20:37:56.128804 4837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:56.129154 4820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 20:37:56.129256 4820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "c4ff6ec8fdf4439ca3d8966917fe1983"
format_stamp: "Formatted at 2026-06-19 20:37:56 on dist-test-slave-0t40"
server_key: "d2502d79c2cbde4a42219bafc9172bca"
server_key_iv: "9b7264b82308b7d7539bfa40dd503667"
server_key_version: "kuduclusterkey@0"
I20260619 20:37:56.129377 4820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:37:56.158836 4820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:37:56.161171 4820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:37:56.161366 4820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:37:56.161615 4820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:37:56.161967 4820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:37:56.162015 4820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:56.162082 4820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:37:56.162111 4820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:56.170339 4820 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:35757
I20260619 20:37:56.170742 4820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:37:56.173136 4951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:35757 every 8 connection(s)
Jun 19 20:37:56 dist-test-slave-0t40 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901476, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:37:56.179908 4817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:56.171699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:48007 (local address 127.25.62.190:41199)
0619 20:37:56.171745 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:56.171747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:56.171757 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:37:56.172265 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:56.172266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:56.172281 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:56.172331 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:56.172864 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.173591 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.174597 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.174824 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.174904 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:56.177014 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:37:56.177015 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:37:56.177015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:37:56.177022 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:37:56.178590 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:56.178998 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:56.178999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:56.179000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:56.179052 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:56.179287 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:56.179288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:56.179288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:56.179401 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:37:56.179473 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:56.179724 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 20:37:56.179809 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:37:56.180284 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 4820
I20260619 20:37:56.180406 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:37:56.180588 25850 external_mini_cluster.cc:1428] Setting key f87a0753e8e1f460680bb185e33d01e0
I20260619 20:37:56.180763 4954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:56.171463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:41199 (local address 127.25.62.130:48007)
0619 20:37:56.171620 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:56.171661 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:56.172153 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:56.172401 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:56.172405 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:56.172584 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:56.172766 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:56.172770 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.173691 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.173692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:56.174481 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:56.174483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.174912 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.174913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:56.175029 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:56.175523 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:56.175531 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:56.176907 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:37:56.178688 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:37:56.178695 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:37:56.178698 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:37:56.178913 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:37:56.179114 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:37:56.179114 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:37:56.179115 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:37:56.179224 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:37:56.179463 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:37:56.179469 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:37:56.179602 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 20:37:56.179705 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 20:37:56.179871 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260619 20:37:56.181520 4952 heartbeater.cc:347] Connected to a master server at 127.25.62.190:41199
I20260619 20:37:56.181605 4952 heartbeater.cc:464] Registering TS with master...
I20260619 20:37:56.181805 4952 heartbeater.cc:511] Master 127.25.62.190:41199 requested a full tablet report, sending...
I20260619 20:37:56.182219 4602 ts_manager.cc:194] Registered new tserver with Master: c4ff6ec8fdf4439ca3d8966917fe1983 (127.25.62.130:35757)
I20260619 20:37:56.182700 4602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:48007
WARNING: no policy specified for kudu/127.25.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
I20260619 20:37:56.211992 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.62.148:43443/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41199
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:42827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:37:56.303941 4957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:37:56.304171 4957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:37:56.304211 4957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:37:56.305502 4957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:37:56.305562 4957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:37:56.305606 4957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:37:56.307190 4957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:42827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.25.62.148:43443/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:37:56.307447 4957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:37:56.307711 4957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:37:56.312712 4965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:37:56.312790 4962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:56.313184 4957 server_base.cc:1034] running on GCE node
W20260619 20:37:56.313544 4963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:37:56.314800 4957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:37:56.315092 4957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:37:56.316269 4957 hybrid_clock.cc:648] HybridClock initialized: now 1781901476316246 us; error 38 us; skew 500 ppm
Jun 19 20:37:56 dist-test-slave-0t40 krb5kdc[4125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901476, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:37:56.320106 4957 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:37:56.320645 4957 webserver.cc:460] Webserver started at http://127.25.62.131:44903/ using document root <none> and password file <none>
I20260619 20:37:56.320936 4957 fs_manager.cc:359] Metadata directory not provided
I20260619 20:37:56.321002 4957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:37:56.321156 4957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:37:56 dist-test-slave-0t40 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901476, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
I20260619 20:37:56.335559 4957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "76b35e5b1b094d23957054d46a70c4b1"
format_stamp: "Formatted at 2026-06-19 20:37:56 on dist-test-slave-0t40"
server_key: "d592093a0e77b5c819b9b0572efe747b"
server_key_iv: "653399c564fb1bba4881cb157bf58680"
server_key_version: "kuduclusterkey@0"
I20260619 20:37:56.336169 4957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "76b35e5b1b094d23957054d46a70c4b1"
format_stamp: "Formatted at 2026-06-19 20:37:56 on dist-test-slave-0t40"
server_key: "d592093a0e77b5c819b9b0572efe747b"
server_key_iv: "653399c564fb1bba4881cb157bf58680"
server_key_version: "kuduclusterkey@0"
I20260619 20:37:56.342655 4957 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.000s
I20260619 20:37:56.364351 4972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:56.364537 4957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:37:56.364621 4957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "76b35e5b1b094d23957054d46a70c4b1"
format_stamp: "Formatted at 2026-06-19 20:37:56 on dist-test-slave-0t40"
server_key: "d592093a0e77b5c819b9b0572efe747b"
server_key_iv: "653399c564fb1bba4881cb157bf58680"
server_key_version: "kuduclusterkey@0"
I20260619 20:37:56.364717 4957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:37:56.422305 4957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:37:56.424907 4957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:37:56.425161 4957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:37:56.425482 4957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:37:56.425897 4957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:37:56.425994 4957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:56.426045 4957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:37:56.426066 4957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:37:56.432761 4957 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:34487
I20260619 20:37:56.433293 4957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:37:56.437487 5085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:34487 every 8 connection(s)
I20260619 20:37:56.441087 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 4957
I20260619 20:37:56.441184 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:37:56.441362 25850 external_mini_cluster.cc:1428] Setting key ffb82310245d9fe233939a7d04d45e51
Jun 19 20:37:56 dist-test-slave-0t40 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901476, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:37:56.450253 4817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:56.439576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:49583 (local address 127.25.62.190:41199)
0619 20:37:56.439621 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:56.439623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:56.439882 (+ 259us) server_negotiation.cc:407] Connection header received
0619 20:37:56.440560 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:56.440561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:56.440575 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:56.440607 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:56.441477 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.442252 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.443778 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.444063 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.444110 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:56.446834 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:37:56.446836 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:37:56.446836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:37:56.446844 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:37:56.448594 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:56.449140 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:56.449141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:56.449141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:56.449201 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:56.449473 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:56.449474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:56.449475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:56.449606 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:37:56.449656 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:56.450071 (+ 415us) server_negotiation.cc:299] Negotiation successful
0619 20:37:56.450155 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:37:56.451025 5088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:56.439553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:41199 (local address 127.25.62.131:49583)
0619 20:37:56.439725 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:56.439776 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:56.440426 (+ 650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:56.440799 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:56.440804 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:56.441079 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:37:56.441354 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:56.441371 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.442398 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.442399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:56.443674 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:56.443677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.444203 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.444204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:56.444347 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:56.444982 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:37:56.444993 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:37:56.446707 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:37:56.448693 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:37:56.448701 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:37:56.448703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:37:56.449007 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:37:56.449271 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:37:56.449271 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:37:56.449272 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:37:56.449399 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:37:56.449686 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:37:56.449691 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:37:56.449864 (+ 173us) client_negotiation.cc:769] Sending connection context
0619 20:37:56.450616 (+ 752us) client_negotiation.cc:240] Negotiation successful
0619 20:37:56.450781 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260619 20:37:56.451751 5086 heartbeater.cc:347] Connected to a master server at 127.25.62.190:41199
I20260619 20:37:56.451850 5086 heartbeater.cc:464] Registering TS with master...
I20260619 20:37:56.452036 5086 heartbeater.cc:511] Master 127.25.62.190:41199 requested a full tablet report, sending...
I20260619 20:37:56.453017 4602 ts_manager.cc:194] Registered new tserver with Master: 76b35e5b1b094d23957054d46a70c4b1 (127.25.62.131:34487)
I20260619 20:37:56.453538 4602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:49583
I20260619 20:37:56.463661 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:37:56.477339 4817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:56.468421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43190 (local address 127.25.62.190:41199)
0619 20:37:56.468477 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:56.468479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:56.468693 (+ 214us) server_negotiation.cc:407] Connection header received
0619 20:37:56.468786 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:56.468787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:56.468800 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:56.468860 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:37:56.469950 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.470733 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.471917 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.472199 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.472267 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:56.473907 (+ 1640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:37:56.473909 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:37:56.473909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:37:56.473917 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:37:56.475871 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:56.476298 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:56.476299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:56.476299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:56.476371 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:37:56.476604 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:37:56.476605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:37:56.476605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:37:56.476748 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:37:56.476865 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:56.476991 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 20:37:56.477105 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 20:37:56.479386 4602 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:37:56.481900 4602 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:37:56.491852 5100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:56.485596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:33443 (local address 127.0.0.1:56190)
0619 20:37:56.485860 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:56.485904 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:56.486675 (+ 771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:56.486990 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:56.486994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:56.486996 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:37:56.487063 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:56.487065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.488772 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.488773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:56.490137 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:56.490140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.491542 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.491543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:56.491628 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:56.491642 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 20:37:56.491713 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 20:37:56.491764 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":73,"threads_started":1}
I20260619 20:37:56.492538 5103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:56.486004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56190 (local address 127.25.62.129:33443)
0619 20:37:56.486127 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:56.486133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:56.486144 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:37:56.486791 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:56.486794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:56.486811 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:56.486904 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:37:56.487161 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.488678 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.490251 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.491369 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.491445 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:56.491505 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:56.492407 (+ 902us) server_negotiation.cc:299] Negotiation successful
0619 20:37:56.492452 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":65,"threads_started":1}
I20260619 20:37:56.493407 4741 tablet_service.cc:1467] Processing CreateTablet for tablet 9d1ee3e04e604e2c9fb02016969c353f (DEFAULT_TABLE table=test-table [id=127b6547a88141faa18f5da38d013724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:37:56.493777 4741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d1ee3e04e604e2c9fb02016969c353f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:37:56.496665 5105 tablet_bootstrap.cc:492] T 9d1ee3e04e604e2c9fb02016969c353f P a0e6a88c02ba43ceb2a65f828e78d49f: Bootstrap starting.
I20260619 20:37:56.497525 5105 tablet_bootstrap.cc:654] T 9d1ee3e04e604e2c9fb02016969c353f P a0e6a88c02ba43ceb2a65f828e78d49f: Neither blocks nor log segments found. Creating new log.
I20260619 20:37:56.497741 5100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:56.485814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:34487 (local address 127.0.0.1:42824)
0619 20:37:56.492258 (+ 6444us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:56.492268 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:56.492323 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:56.493228 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:56.493229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:56.493231 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:37:56.493263 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:56.493264 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.494908 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.494909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:56.496016 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:56.496018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.497526 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.497527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:56.497590 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:56.497595 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:37:56.497631 (+ 36us) client_negotiation.cc:240] Negotiation successful
0619 20:37:56.497665 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6432,"thread_start_us":65,"threads_started":1}
I20260619 20:37:56.497903 5105 log.cc:824] T 9d1ee3e04e604e2c9fb02016969c353f P a0e6a88c02ba43ceb2a65f828e78d49f: Log is configured to *not* fsync() on all Append() calls
I20260619 20:37:56.498348 5104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:56.486184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42824 (local address 127.25.62.131:34487)
0619 20:37:56.486368 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:56.486372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:56.492967 (+ 6595us) server_negotiation.cc:407] Connection header received
0619 20:37:56.493019 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:56.493021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:56.493056 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:56.493143 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:37:56.493633 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.494796 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.496174 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.497428 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.498099 (+ 671us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:56.498186 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:56.498213 (+ 27us) server_negotiation.cc:299] Negotiation successful
0619 20:37:56.498253 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":89,"threads_started":1}
I20260619 20:37:56.498708 5105 tablet_bootstrap.cc:492] T 9d1ee3e04e604e2c9fb02016969c353f P a0e6a88c02ba43ceb2a65f828e78d49f: No bootstrap required, opened a new log
I20260619 20:37:56.498775 5105 ts_tablet_manager.cc:1397] T 9d1ee3e04e604e2c9fb02016969c353f P a0e6a88c02ba43ceb2a65f828e78d49f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 20:37:56.499101 5020 tablet_service.cc:1467] Processing CreateTablet for tablet 9d1ee3e04e604e2c9fb02016969c353f (DEFAULT_TABLE table=test-table [id=127b6547a88141faa18f5da38d013724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:37:56.499437 5020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d1ee3e04e604e2c9fb02016969c353f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:37:56.501215 5102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:56.485632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:35757 (local address 127.0.0.1:49212)
0619 20:37:56.486818 (+ 1186us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:56.486828 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:56.486870 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:56.497448 (+ 10578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:56.497449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:56.497451 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:37:56.497499 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:56.497501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.498767 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.498768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:56.499833 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:56.499836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.500932 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.500933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:56.500996 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:56.501001 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:37:56.501088 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 20:37:56.501132 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1169,"thread_start_us":92,"threads_started":1}
I20260619 20:37:56.501531 5101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:56.485749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49212 (local address 127.25.62.130:35757)
0619 20:37:56.497148 (+ 11399us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:56.497153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:56.497166 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:37:56.497207 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:56.497209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:56.497226 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:56.497321 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:37:56.497586 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.498656 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.500006 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.500720 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.500819 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:56.500893 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:56.501397 (+ 504us) server_negotiation.cc:299] Negotiation successful
0619 20:37:56.501439 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11352,"thread_start_us":89,"threads_started":1}
I20260619 20:37:56.501425 5105 raft_consensus.cc:348] T 9d1ee3e04e604e2c9fb02016969c353f P a0e6a88c02ba43ceb2a65f828e78d49f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e6a88c02ba43ceb2a65f828e78d49f" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33443 } } peers { permanent_uuid: "76b35e5b1b094d23957054d46a70c4b1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34487 } } peers { permanent_uuid: "c4ff6ec8fdf4439ca3d8966917fe1983" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35757 } }
I20260619 20:37:56.501579 5105 raft_consensus.cc:374] T 9d1ee3e04e604e2c9fb02016969c353f P a0e6a88c02ba43ceb2a65f828e78d49f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:37:56.501614 5105 raft_consensus.cc:729] T 9d1ee3e04e604e2c9fb02016969c353f P a0e6a88c02ba43ceb2a65f828e78d49f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e6a88c02ba43ceb2a65f828e78d49f, State: Initialized, Role: FOLLOWER
I20260619 20:37:56.502059 5105 consensus_queue.cc:260] T 9d1ee3e04e604e2c9fb02016969c353f P a0e6a88c02ba43ceb2a65f828e78d49f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e6a88c02ba43ceb2a65f828e78d49f" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33443 } } peers { permanent_uuid: "76b35e5b1b094d23957054d46a70c4b1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34487 } } peers { permanent_uuid: "c4ff6ec8fdf4439ca3d8966917fe1983" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35757 } }
I20260619 20:37:56.502154 4886 tablet_service.cc:1467] Processing CreateTablet for tablet 9d1ee3e04e604e2c9fb02016969c353f (DEFAULT_TABLE table=test-table [id=127b6547a88141faa18f5da38d013724]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:37:56.502424 5105 ts_tablet_manager.cc:1428] T 9d1ee3e04e604e2c9fb02016969c353f P a0e6a88c02ba43ceb2a65f828e78d49f: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260619 20:37:56.502492 4886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d1ee3e04e604e2c9fb02016969c353f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:37:56.503219 5108 tablet_bootstrap.cc:492] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1: Bootstrap starting.
I20260619 20:37:56.504145 5108 tablet_bootstrap.cc:654] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1: Neither blocks nor log segments found. Creating new log.
I20260619 20:37:56.504586 5108 log.cc:824] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1: Log is configured to *not* fsync() on all Append() calls
I20260619 20:37:56.505125 4815 heartbeater.cc:503] Master 127.25.62.190:41199 was elected leader, sending a full tablet report...
I20260619 20:37:56.505419 5108 tablet_bootstrap.cc:492] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1: No bootstrap required, opened a new log
I20260619 20:37:56.505543 5108 ts_tablet_manager.cc:1397] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:37:56.505844 5110 tablet_bootstrap.cc:492] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983: Bootstrap starting.
I20260619 20:37:56.507411 5110 tablet_bootstrap.cc:654] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983: Neither blocks nor log segments found. Creating new log.
I20260619 20:37:56.507706 5108 raft_consensus.cc:348] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e6a88c02ba43ceb2a65f828e78d49f" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33443 } } peers { permanent_uuid: "76b35e5b1b094d23957054d46a70c4b1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34487 } } peers { permanent_uuid: "c4ff6ec8fdf4439ca3d8966917fe1983" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35757 } }
I20260619 20:37:56.508001 5108 raft_consensus.cc:374] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:37:56.508095 5108 raft_consensus.cc:729] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76b35e5b1b094d23957054d46a70c4b1, State: Initialized, Role: FOLLOWER
I20260619 20:37:56.508276 5108 consensus_queue.cc:260] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e6a88c02ba43ceb2a65f828e78d49f" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33443 } } peers { permanent_uuid: "76b35e5b1b094d23957054d46a70c4b1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34487 } } peers { permanent_uuid: "c4ff6ec8fdf4439ca3d8966917fe1983" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35757 } }
I20260619 20:37:56.508600 5108 ts_tablet_manager.cc:1428] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:37:56.508749 5086 heartbeater.cc:503] Master 127.25.62.190:41199 was elected leader, sending a full tablet report...
I20260619 20:37:56.509682 5110 log.cc:824] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983: Log is configured to *not* fsync() on all Append() calls
I20260619 20:37:56.510489 5110 tablet_bootstrap.cc:492] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983: No bootstrap required, opened a new log
I20260619 20:37:56.510564 5110 ts_tablet_manager.cc:1397] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260619 20:37:56.512560 5110 raft_consensus.cc:348] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e6a88c02ba43ceb2a65f828e78d49f" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33443 } } peers { permanent_uuid: "76b35e5b1b094d23957054d46a70c4b1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34487 } } peers { permanent_uuid: "c4ff6ec8fdf4439ca3d8966917fe1983" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35757 } }
I20260619 20:37:56.512738 5110 raft_consensus.cc:374] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:37:56.512822 5110 raft_consensus.cc:729] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4ff6ec8fdf4439ca3d8966917fe1983, State: Initialized, Role: FOLLOWER
I20260619 20:37:56.512948 5110 consensus_queue.cc:260] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e6a88c02ba43ceb2a65f828e78d49f" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33443 } } peers { permanent_uuid: "76b35e5b1b094d23957054d46a70c4b1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34487 } } peers { permanent_uuid: "c4ff6ec8fdf4439ca3d8966917fe1983" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35757 } }
I20260619 20:37:56.513315 5110 ts_tablet_manager.cc:1428] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:37:56.513478 4952 heartbeater.cc:503] Master 127.25.62.190:41199 was elected leader, sending a full tablet report...
Jun 19 20:37:56 dist-test-slave-0t40 krb5kdc[4125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901476, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
I20260619 20:37:56.566486 5113 raft_consensus.cc:482] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:37:56.566624 5113 raft_consensus.cc:504] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e6a88c02ba43ceb2a65f828e78d49f" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33443 } } peers { permanent_uuid: "76b35e5b1b094d23957054d46a70c4b1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34487 } } peers { permanent_uuid: "c4ff6ec8fdf4439ca3d8966917fe1983" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35757 } }
I20260619 20:37:56.567003 5113 leader_election.cc:288] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0e6a88c02ba43ceb2a65f828e78d49f (127.25.62.129:33443), 76b35e5b1b094d23957054d46a70c4b1 (127.25.62.131:34487)
I20260619 20:37:56.571429 4954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:56.567167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:33443 (local address 127.25.62.130:41177)
0619 20:37:56.567468 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:56.567479 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:56.567523 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:56.568007 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:56.568009 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:56.568011 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:37:56.568085 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:56.568087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.569111 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.569112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:56.570310 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:56.570312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.571166 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.571167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:56.571251 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:56.571255 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:37:56.571309 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 20:37:56.571349 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282}
I20260619 20:37:56.572057 5103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:56.567676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:41177 (local address 127.25.62.129:33443)
0619 20:37:56.567796 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:56.567799 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:56.567807 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:37:56.567836 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:56.567837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:56.567849 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:56.567908 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:37:56.568168 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.568950 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.570423 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.571044 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.571121 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:56.571131 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:56.571946 (+ 815us) server_negotiation.cc:299] Negotiation successful
0619 20:37:56.571987 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260619 20:37:56.572306 4768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d1ee3e04e604e2c9fb02016969c353f" candidate_uuid: "c4ff6ec8fdf4439ca3d8966917fe1983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e6a88c02ba43ceb2a65f828e78d49f" is_pre_election: true
I20260619 20:37:56.572470 4768 raft_consensus.cc:2454] T 9d1ee3e04e604e2c9fb02016969c353f P a0e6a88c02ba43ceb2a65f828e78d49f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4ff6ec8fdf4439ca3d8966917fe1983 in term 0.
I20260619 20:37:56.572916 4839 leader_election.cc:302] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0e6a88c02ba43ceb2a65f828e78d49f, c4ff6ec8fdf4439ca3d8966917fe1983; no voters:
I20260619 20:37:56.573065 5113 raft_consensus.cc:2781] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:37:56.573110 5113 raft_consensus.cc:482] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:37:56.573135 5113 raft_consensus.cc:3037] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:37:56.574050 5113 raft_consensus.cc:504] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e6a88c02ba43ceb2a65f828e78d49f" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33443 } } peers { permanent_uuid: "76b35e5b1b094d23957054d46a70c4b1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34487 } } peers { permanent_uuid: "c4ff6ec8fdf4439ca3d8966917fe1983" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35757 } }
I20260619 20:37:56.574186 5113 leader_election.cc:288] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 [CANDIDATE]: Term 1 election: Requested vote from peers a0e6a88c02ba43ceb2a65f828e78d49f (127.25.62.129:33443), 76b35e5b1b094d23957054d46a70c4b1 (127.25.62.131:34487)
I20260619 20:37:56.574426 4768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d1ee3e04e604e2c9fb02016969c353f" candidate_uuid: "c4ff6ec8fdf4439ca3d8966917fe1983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e6a88c02ba43ceb2a65f828e78d49f"
I20260619 20:37:56.574503 4768 raft_consensus.cc:3037] T 9d1ee3e04e604e2c9fb02016969c353f P a0e6a88c02ba43ceb2a65f828e78d49f [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:37:56.575511 4768 raft_consensus.cc:2454] T 9d1ee3e04e604e2c9fb02016969c353f P a0e6a88c02ba43ceb2a65f828e78d49f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4ff6ec8fdf4439ca3d8966917fe1983 in term 1.
I20260619 20:37:56.575722 4839 leader_election.cc:302] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0e6a88c02ba43ceb2a65f828e78d49f, c4ff6ec8fdf4439ca3d8966917fe1983; no voters:
I20260619 20:37:56.576717 5113 raft_consensus.cc:2781] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:37:56.576911 5113 raft_consensus.cc:686] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 [term 1 LEADER]: Becoming Leader. State: Replica: c4ff6ec8fdf4439ca3d8966917fe1983, State: Running, Role: LEADER
I20260619 20:37:56.577005 5113 consensus_queue.cc:237] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e6a88c02ba43ceb2a65f828e78d49f" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33443 } } peers { permanent_uuid: "76b35e5b1b094d23957054d46a70c4b1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34487 } } peers { permanent_uuid: "c4ff6ec8fdf4439ca3d8966917fe1983" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35757 } }
I20260619 20:37:56.577947 4602 catalog_manager.cc:5495] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4ff6ec8fdf4439ca3d8966917fe1983 (127.25.62.130). New cstate: current_term: 1 leader_uuid: "c4ff6ec8fdf4439ca3d8966917fe1983" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0e6a88c02ba43ceb2a65f828e78d49f" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76b35e5b1b094d23957054d46a70c4b1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4ff6ec8fdf4439ca3d8966917fe1983" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35757 } health_report { overall_health: HEALTHY } } }
I20260619 20:37:56.578668 4954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:56.567263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:34487 (local address 127.25.62.130:60567)
0619 20:37:56.571506 (+ 4243us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:56.571516 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:37:56.571558 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:37:56.575684 (+ 4126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:37:56.575685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:37:56.575687 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:37:56.575726 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:56.575727 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.576698 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.576699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:56.577721 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:37:56.577724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.578431 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.578432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:37:56.578491 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:56.578494 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:37:56.578566 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 20:37:56.578603 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4232,"thread_start_us":109,"threads_started":1}
I20260619 20:37:56.579046 5104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:56.567732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:60567 (local address 127.25.62.131:34487)
0619 20:37:56.567971 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:56.567973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:56.571715 (+ 3742us) server_negotiation.cc:407] Connection header received
0619 20:37:56.571746 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:56.571747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:56.571761 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:56.571828 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:37:56.575844 (+ 4016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.576587 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.577822 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.578356 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.578430 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:56.578439 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:56.578932 (+ 493us) server_negotiation.cc:299] Negotiation successful
0619 20:37:56.578981 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215}
I20260619 20:37:56.579353 5039 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d1ee3e04e604e2c9fb02016969c353f" candidate_uuid: "c4ff6ec8fdf4439ca3d8966917fe1983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76b35e5b1b094d23957054d46a70c4b1"
I20260619 20:37:56.579375 5040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d1ee3e04e604e2c9fb02016969c353f" candidate_uuid: "c4ff6ec8fdf4439ca3d8966917fe1983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76b35e5b1b094d23957054d46a70c4b1" is_pre_election: true
I20260619 20:37:56.579519 5040 raft_consensus.cc:2454] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4ff6ec8fdf4439ca3d8966917fe1983 in term 0.
I20260619 20:37:56.608011 5101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:56.604463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49228 (local address 127.25.62.130:35757)
0619 20:37:56.604508 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:37:56.604510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:37:56.604518 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:37:56.604562 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:37:56.604562 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:37:56.604585 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:37:56.604654 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:37:56.604882 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.605665 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.606806 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:37:56.607125 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:37:56.607189 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:37:56.607399 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:37:56.607537 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:37:56.607621 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 20:37:56.607884 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 20:37:56.607906 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:37:56.610387 5040 raft_consensus.cc:3037] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:37:56.610459 4768 raft_consensus.cc:1261] T 9d1ee3e04e604e2c9fb02016969c353f P a0e6a88c02ba43ceb2a65f828e78d49f [term 1 FOLLOWER]: Refusing update from remote peer c4ff6ec8fdf4439ca3d8966917fe1983: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:37:56.610793 5115 consensus_queue.cc:1035] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e6a88c02ba43ceb2a65f828e78d49f" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:37:56.611408 5040 raft_consensus.cc:1261] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1 [term 1 FOLLOWER]: Refusing update from remote peer c4ff6ec8fdf4439ca3d8966917fe1983: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:37:56.611845 5115 consensus_queue.cc:1035] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76b35e5b1b094d23957054d46a70c4b1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:37:56.615489 5118 mvcc.cc:205] Tried to move back new op lower bound from 7298668448192585728 to 7298668448060010496. Current Snapshot: MvccSnapshot[applied={T|T < 7298668448192585728}]
I20260619 20:37:56.616111 5119 mvcc.cc:205] Tried to move back new op lower bound from 7298668448192585728 to 7298668448060010496. Current Snapshot: MvccSnapshot[applied={T|T < 7298668448192585728}]
I20260619 20:37:56.618584 4602 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:37:56.618681 4602 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:37:56.619436 4602 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 101fc7bfc6744d62be14c16ccbc1be5f: Sending DeleteTablet for 3 replicas of tablet 9d1ee3e04e604e2c9fb02016969c353f
I20260619 20:37:56.619823 5020 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d1ee3e04e604e2c9fb02016969c353f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:37:56 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:42824
I20260619 20:37:56.619916 4886 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d1ee3e04e604e2c9fb02016969c353f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:37:56 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:49212
I20260619 20:37:56.619983 4741 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d1ee3e04e604e2c9fb02016969c353f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:37:56 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:56190
I20260619 20:37:56.620471 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 4683
I20260619 20:37:56.621723 5126 tablet_replica.cc:330] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1: stopping tablet replica
I20260619 20:37:56.621832 5126 raft_consensus.cc:2229] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:37:56.621896 5126 pending_rounds.cc:62] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1: Trying to abort 2 pending ops.
I20260619 20:37:56.621922 5126 pending_rounds.cc:68] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7298668448192585728 op_type: WRITE_OP write_request { tablet_id: "9d1ee3e04e604e2c9fb02016969c353f" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\320\323\326\321\006\"8\n\ntest-admin\022*\n 127b6547a88141faa18f5da38d013724\020\001\030\001 \001(\001" signature: "<redacted>""?\005\374Y\313^xA\020\365\327\360\025\003E\370\250f\3402\361q\n\004Va\2271<\276\326\236~M\034=\340\334\317\246\\\226\353\366\300\273\037\353z\030\305\352\361\241\376\331\207\000&e\312\307Om" signing_key_seq_num: 0 } } request_id { client_id: "9442229537fb47f499476a7ad3673be1" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260619 20:37:56.622107 5126 pending_rounds.cc:68] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298668448060010496 op_type: NO_OP noop_request { }
I20260619 20:37:56.622150 5126 raft_consensus.cc:2866] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260619 20:37:56.622176 5126 raft_consensus.cc:2258] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 20:37:56.624773 4839 consensus_peers.cc:487] T 9d1ee3e04e604e2c9fb02016969c353f P c4ff6ec8fdf4439ca3d8966917fe1983 -> Peer 76b35e5b1b094d23957054d46a70c4b1 (127.25.62.131:34487): Couldn't send request to peer 76b35e5b1b094d23957054d46a70c4b1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260619 20:37:56.628309 5126 ts_tablet_manager.cc:1905] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 20:37:56.628990 4587 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.62.129:33443 (error 108)
I20260619 20:37:56.629654 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 4820
I20260619 20:37:56.630550 5126 ts_tablet_manager.cc:1918] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:37:56.630622 5126 log.cc:1196] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/9d1ee3e04e604e2c9fb02016969c353f
I20260619 20:37:56.631212 5126 ts_tablet_manager.cc:1939] T 9d1ee3e04e604e2c9fb02016969c353f P 76b35e5b1b094d23957054d46a70c4b1: Deleting consensus metadata
I20260619 20:37:56.633842 4587 catalog_manager.cc:4841] TS 76b35e5b1b094d23957054d46a70c4b1 (127.25.62.131:34487): tablet 9d1ee3e04e604e2c9fb02016969c353f (table test-table [id=127b6547a88141faa18f5da38d013724]) successfully deleted
I20260619 20:37:56.641184 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 4957
I20260619 20:37:56.642181 5102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:56.642046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:35757 (local address 127.0.0.1:49230)
0619 20:37:56.642103 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:56.642145 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.130:35757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 20:37:56.642568 5102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:37:56.642484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:33443 (local address 127.0.0.1:56200)
0619 20:37:56.642522 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 20:37:56.642543 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.129:33443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260619 20:37:56.642789 4588 catalog_manager.cc:4584] TS c4ff6ec8fdf4439ca3d8966917fe1983 (127.25.62.130:35757): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9d1ee3e04e604e2c9fb02016969c353f: Network error: Client connection negotiation failed: client connection to 127.25.62.130:35757: connect: Connection refused (error 111)
I20260619 20:37:56.646747 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 4570
I20260619 20:37:56.652757 25850 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 20:37:56.997148 25850 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 20:37:56.997246 25850 mini_ranger.cc:66] Stopping Ranger...
I20260619 20:37:57.341475 25850 mini_ranger.cc:68] Stopped Ranger
2026-06-19 20:37:57.341 UTC [4155] LOG: received smart shutdown request
2026-06-19 20:37:57.344 UTC [4155] LOG: background worker "logical replication launcher" (PID 4164) exited with exit code 1
2026-06-19 20:37:57.345 UTC [4159] LOG: shutting down
2026-06-19 20:37:57.501 UTC [4155] LOG: database system is shut down
2026-06-19T20:37:57Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45527 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:37:57 dist-test-slave-0t40 krb5kdc[5138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:37:57 dist-test-slave-0t40 krb5kdc[5138](info): set up 2 sockets
Jun 19 20:37:57 dist-test-slave-0t40 krb5kdc[5138](info): commencing operation
krb5kdc: starting...
W20260619 20:37:58.629612 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:37:58 dist-test-slave-0t40 krb5kdc[5138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:37:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:37:58Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.25.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.25.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.25.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/rangeradmin_127.25.62.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.25.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/rangeradmin_127.25.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.25.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.25.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.25.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/rangerlookup_127.25.62.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.25.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/rangerlookup_127.25.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.25.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/HTTP_127.25.62.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/HTTP_127.25.62.148@KRBTEST.COM.keytab.
I20260619 20:37:58.795488 25850 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskXT8PDl/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/postgres -l logfile start
2026-06-19 20:38:01.290 UTC [5168] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 20:38:01.290 UTC [5168] LOG: listening on IPv4 address "127.25.62.148", port 35803
2026-06-19 20:38:01.294 UTC [5168] LOG: listening on Unix socket "/tmp/.s.PGSQL.35803"
2026-06-19 20:38:01.331 UTC [5171] LOG: database system was shut down at 2026-06-19 20:37:59 UTC
2026-06-19 20:38:01.335 UTC [5168] LOG: database system is ready to accept connections
I20260619 20:38:02.281206 25850 mini_postgres.cc:96] Postgres bound to 35803
2026-06-19 20:38:02.286 UTC [5179] FATAL: database "slave" does not exist
127.25.62.148:35803 - accepting connections
I20260619 20:38:02.287366 25850 mini_ranger.cc:161] Starting Ranger...
I20260619 20:38:02.301815 25850 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 20:38:02.775877 25850 mini_ranger.cc:87] Created ranger Postgres database
I20260619 20:38:02.775959 25850 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ranger-admin
2026-06-19 20:38:03,044 [I] DB FLAVOR :POSTGRES
2026-06-19 20:38:03,044 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 20:38:03,044 [I] Checking connection..
2026-06-19 20:38:03,044 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 20:38:03,300 [I] Checking connection passed.
2026-06-19 20:38:03,301 [I] --------- Verifying version history table ---------
2026-06-19 20:38:03,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 20:38:03,540 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 20:38:03,541 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 20:38:03,779 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 20:38:03,779 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 20:38:03,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 20:38:04.024 UTC [5243] WARNING: there is no transaction in progress
2026-06-19 20:38:04,030 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 20:38:04,030 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 20:38:04,278 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 20:38:04,279 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 20:38:04,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 20:38:04,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 20:38:04,755 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-0t40.c.gcp-upstream.internal','N') ;"
2026-06-19 20:38:04,996 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 20:38:05,255 [I] Table x_portal_user does not exist in database ranger
2026-06-19 20:38:05,255 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 20:38:05,515 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 20:38:05,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 20:38:05,739 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 20:38:05,988 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 20:38:05,988 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 20:38:06.964 UTC [5351] WARNING: there is no transaction in progress
2026-06-19 20:38:06.980 UTC [5351] WARNING: there is no transaction in progress
2026-06-19 20:38:07.000 UTC [5351] WARNING: there is no transaction in progress
2026-06-19 20:38:07.017 UTC [5351] WARNING: there is no transaction in progress
2026-06-19 20:38:07.036 UTC [5351] WARNING: there is no transaction in progress
2026-06-19 20:38:07.054 UTC [5351] WARNING: there is no transaction in progress
2026-06-19 20:38:07.107 UTC [5351] WARNING: there is no transaction in progress
2026-06-19 20:38:07.114 UTC [5351] WARNING: there is no transaction in progress
2026-06-19 20:38:07.128 UTC [5351] WARNING: there is no transaction in progress
2026-06-19 20:38:07.139 UTC [5351] WARNING: there is no transaction in progress
2026-06-19 20:38:07.149 UTC [5351] WARNING: there is no transaction in progress
2026-06-19 20:38:07.164 UTC [5351] WARNING: there is no transaction in progress
2026-06-19 20:38:07.173 UTC [5351] WARNING: there is no transaction in progress
2026-06-19 20:38:07.179 UTC [5351] WARNING: there is no transaction in progress
2026-06-19 20:38:07.186 UTC [5351] WARNING: there is no transaction in progress
2026-06-19 20:38:07.192 UTC [5351] WARNING: there is no transaction in progress
2026-06-19 20:38:07,870 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 20:38:07,871 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 20:38:08,108 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 20:38:08,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 20:38:08,340 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 20:38:08,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 20:38:08,567 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 20:38:08,568 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 20:38:08,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 20:38:09,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-0t40.c.gcp-upstream.internal';"
2026-06-19 20:38:09,305 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 20:38:09,306 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 20:38:09,563 [I] DB_PATCHES have already been applied
I20260619 20:38:09.569139 25850 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ranger-admin:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskXT8PDl/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 20:38:09.569207 25850 mini_ranger.cc:193] Using host: 127.25.62.148
I20260619 20:38:09.572211 25850 mini_ranger.cc:227] Ranger admin URL: http://127.25.62.148:54281
Jun 19, 2026 8:38:09 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:38:09 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:38:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 8:38:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 8:38:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 8:38:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 8:38:10 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 20:38:10.659351 25850 mini_ranger.cc:160] Time spent starting Ranger: real 8.372s user 0.003s sys 0.005s
Jun 19, 2026 8:38:29 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 8:38:29 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 8:38:29 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 8:38:29 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 8:38:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 8:38:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 8:38:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 8:38:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 8:38:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 8:38:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 8:38:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 8:38:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 8:38:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 8:38:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 8:38:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 8:38:29 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 8:38:29 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 8:38:30 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 20:38:31.316475 25850 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.25.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.25.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.25.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/rangerkms_127.25.62.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.25.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/rangerkms_127.25.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.25.62.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.25.62.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.25.62.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/HTTP_127.25.62.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.25.62.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/HTTP_127.25.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 20:38:31 dist-test-slave-0t40 krb5kdc[5138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901511, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 20:38:31.397425 25850 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 20:38:31.407238 25850 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 20:38:32.282059 25850 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 20:38:32.282146 25850 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ranger-kms
I20260619 20:38:32.282168 25850 mini_ranger_kms.cc:227] Using postgres at 127.25.62.148:35803
2026-06-19 20:38:32,443 [I] DB FLAVOR :POSTGRES
2026-06-19 20:38:32,443 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 20:38:32,443 [I] Checking connection
2026-06-19 20:38:32,443 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 20:38:32,694 [I] connection success
2026-06-19 20:38:32,694 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 20:38:32,694 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 20:38:32,694 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 20:38:32,930 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 20:38:32,931 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 20:38:32,931 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 20:38:32,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.25.62.148:35803/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 20:38:33,178 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 20:38:33.617700 25850 mini_ranger_kms.cc:310] Created kms service
I20260619 20:38:33.816807 25850 mini_ranger_kms.cc:326] Created kudu user
I20260619 20:38:33.857725 25850 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 20:38:33 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901511, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
I20260619 20:38:34.257058 25850 mini_ranger_kms.cc:384] Added ranger policy
I20260619 20:38:34.257174 25850 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ranger-kms:/tmp/dist-test-taskXT8PDl/build/release/bin/postgresql.jar:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskXT8PDl/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 20:38:34.257195 25850 mini_ranger_kms.cc:240] Using host: 127.25.62.148
I20260619 20:38:34.260490 25850 mini_ranger_kms.cc:276] Ranger KMS PID: 5531
I20260619 20:38:34.260565 25850 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.25.62.148:34895
Jun 19, 2026 8:38:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 8:38:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 8:38:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 8:38:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 8:38:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 8:38:35 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 20:38:36.509469 25850 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.112s user 0.014s sys 0.130s
I20260619 20:38:36.509572 25850 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 20:38:36.509618 25850 mini_ranger_kms.cc:401] 127.25.62.148:34895/kms/v1/keys
Jun 19 20:38:36 dist-test-slave-0t40 krb5kdc[5138](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781901516, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.62.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 20:38:37 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901516, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.62.148@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
Jun 19 20:38:37 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901516, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.25.62.148@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
Jun 19 20:38:39 dist-test-slave-0t40 krb5kdc[5138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:38:39.750070 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:42561
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:33621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:42561
--ranger_config_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.62.148:34895/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:38:39.833269 5582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:39.833647 5582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:39.833696 5582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:39.834932 5582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:38:39.835000 5582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:39.835038 5582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:39.835058 5582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:38:39.835076 5582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:38:39.836602 5582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:33621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/wal
--ranger_kms_url=127.25.62.148:34895/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:42561
--ranger_config_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:42561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:39.836853 5582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:39.837118 5582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:39.843082 5587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:39.843093 5588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:39.843199 5590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:39.843210 5582 server_base.cc:1034] running on GCE node
I20260619 20:38:39.843602 5582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:39.843904 5582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:39.845088 5582 hybrid_clock.cc:648] HybridClock initialized: now 1781901519845073 us; error 30 us; skew 500 ppm
Jun 19 20:38:39 dist-test-slave-0t40 krb5kdc[5138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901519, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:39.852039 5582 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:38:39.852523 5582 webserver.cc:460] Webserver started at http://127.25.62.190:46155/ using document root <none> and password file <none>
I20260619 20:38:39.852747 5582 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:39.852809 5582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:39.852967 5582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:38:39 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901519, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
I20260619 20:38:39.910692 5582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "16f8263a2d47427fb492bc7865d46f09"
format_stamp: "Formatted at 2026-06-19 20:38:39 on dist-test-slave-0t40"
server_key: "7af728f283da5b46209c2e543924bed4"
server_key_iv: "5f6400f9b5d32f5f7327431b35bac054"
server_key_version: "kuduclusterkey@0"
I20260619 20:38:39.911094 5582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "16f8263a2d47427fb492bc7865d46f09"
format_stamp: "Formatted at 2026-06-19 20:38:39 on dist-test-slave-0t40"
server_key: "7af728f283da5b46209c2e543924bed4"
server_key_iv: "5f6400f9b5d32f5f7327431b35bac054"
server_key_version: "kuduclusterkey@0"
I20260619 20:38:39.912523 5582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:38:39.926720 5597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:39.926889 5582 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:38:39.926985 5582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "16f8263a2d47427fb492bc7865d46f09"
format_stamp: "Formatted at 2026-06-19 20:38:39 on dist-test-slave-0t40"
server_key: "7af728f283da5b46209c2e543924bed4"
server_key_iv: "5f6400f9b5d32f5f7327431b35bac054"
server_key_version: "kuduclusterkey@0"
I20260619 20:38:39.927083 5582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:39.935204 5582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:39.937269 5582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:39.937433 5582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:39.942257 5582 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:42561
I20260619 20:38:39.942340 5649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:42561 every 8 connection(s)
I20260619 20:38:39.942683 5582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:38:39.943358 5650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:38:39.945636 5650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09: Bootstrap starting.
I20260619 20:38:39.946074 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 5582
I20260619 20:38:39.946188 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:38:39.946260 5650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09: Neither blocks nor log segments found. Creating new log.
I20260619 20:38:39.946410 25850 external_mini_cluster.cc:1428] Setting key 50dd02d8a9f0716c0ab6047e130e94fe
I20260619 20:38:39.946604 5650 log.cc:824] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09: Log is configured to *not* fsync() on all Append() calls
I20260619 20:38:39.947456 5650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09: No bootstrap required, opened a new log
I20260619 20:38:39.949779 5650 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f8263a2d47427fb492bc7865d46f09" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42561 } }
I20260619 20:38:39.949951 5650 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:38:39.949980 5650 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f8263a2d47427fb492bc7865d46f09, State: Initialized, Role: FOLLOWER
I20260619 20:38:39.950089 5650 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f8263a2d47427fb492bc7865d46f09" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42561 } }
I20260619 20:38:39.950142 5650 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:38:39.950173 5650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:38:39.950209 5650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:38:39.950755 5650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f8263a2d47427fb492bc7865d46f09" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42561 } }
I20260619 20:38:39.950881 5650 leader_election.cc:302] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16f8263a2d47427fb492bc7865d46f09; no voters:
I20260619 20:38:39.951043 5650 leader_election.cc:288] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:38:39.951071 5655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:38:39.951267 5655 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [term 1 LEADER]: Becoming Leader. State: Replica: 16f8263a2d47427fb492bc7865d46f09, State: Running, Role: LEADER
I20260619 20:38:39.951376 5650 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:38:39.951401 5655 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f8263a2d47427fb492bc7865d46f09" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42561 } }
I20260619 20:38:39.951745 5655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16f8263a2d47427fb492bc7865d46f09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f8263a2d47427fb492bc7865d46f09" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42561 } } }
I20260619 20:38:39.951896 5655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [sys.catalog]: This master's current role is: LEADER
I20260619 20:38:39.952080 5655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16f8263a2d47427fb492bc7865d46f09. Latest consensus state: current_term: 1 leader_uuid: "16f8263a2d47427fb492bc7865d46f09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f8263a2d47427fb492bc7865d46f09" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42561 } } }
I20260619 20:38:39.952193 5655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [sys.catalog]: This master's current role is: LEADER
I20260619 20:38:39.952358 5663 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:38:39.952901 5663 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:38:39.953207 5650 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Jun 19 20:38:39 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:38:39.959210 5663 catalog_manager.cc:1347] Generated new cluster ID: f9a06cf59bfe45d3adbf90ac081f6100
I20260619 20:38:39.959308 5663 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:38:39.960736 5654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:39.946940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43648 (local address 127.25.62.190:42561)
0619 20:38:39.947071 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:39.947082 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:39.947166 (+ 84us) server_negotiation.cc:407] Connection header received
0619 20:38:39.947754 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:39.947763 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:39.947825 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:39.947968 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:39.948773 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:39.950218 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:39.951245 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:39.951873 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:39.951954 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:39.955161 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:39.955209 (+ 48us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:39.955209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:39.955310 (+ 101us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:39.958609 (+ 3299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:39.959058 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:39.959060 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:39.959060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:39.959159 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:39.959401 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:39.959402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:39.959402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:39.959767 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:39.959898 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:39.960131 (+ 233us) server_negotiation.cc:299] Negotiation successful
0619 20:38:39.960472 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":53,"threads_started":1}
I20260619 20:38:39.987857 5663 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:38:39.988704 5663 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:38:39.997172 5663 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09: Generated new TSK 0
I20260619 20:38:39.997434 5663 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
I20260619 20:38:40.687927 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.62.148:34895/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:42561
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:33621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:38:40.801172 5695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:40.801398 5695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:40.801436 5695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:40.802731 5695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:40.802809 5695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:40.802855 5695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:38:40.804638 5695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:33621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.25.62.148:34895/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:42561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:40.804903 5695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:40.805207 5695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:40.811132 5702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:40.811352 5695 server_base.cc:1034] running on GCE node
W20260619 20:38:40.811436 5701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:40.811662 5704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:40.811946 5695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:40.812224 5695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:40.813377 5695 hybrid_clock.cc:648] HybridClock initialized: now 1781901520813352 us; error 49 us; skew 500 ppm
Jun 19 20:38:40 dist-test-slave-0t40 krb5kdc[5138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901520, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:40.816833 5695 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:38:40.817407 5695 webserver.cc:460] Webserver started at http://127.25.62.129:34871/ using document root <none> and password file <none>
I20260619 20:38:40.817687 5695 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:40.817803 5695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:40.817998 5695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:38:40 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901520, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
I20260619 20:38:40.843449 5695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "3812d55bd209489fb934fcf99b7e2685"
format_stamp: "Formatted at 2026-06-19 20:38:40 on dist-test-slave-0t40"
server_key: "ea14e8a8a6e72eafb00fa504771a8db3"
server_key_iv: "789790e1d8f7c240c023103570e05ace"
server_key_version: "kuduclusterkey@0"
I20260619 20:38:40.843967 5695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "3812d55bd209489fb934fcf99b7e2685"
format_stamp: "Formatted at 2026-06-19 20:38:40 on dist-test-slave-0t40"
server_key: "ea14e8a8a6e72eafb00fa504771a8db3"
server_key_iv: "789790e1d8f7c240c023103570e05ace"
server_key_version: "kuduclusterkey@0"
I20260619 20:38:40.853475 5695 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.003s sys 0.000s
I20260619 20:38:40.875516 5712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:40.875767 5695 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260619 20:38:40.875859 5695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "3812d55bd209489fb934fcf99b7e2685"
format_stamp: "Formatted at 2026-06-19 20:38:40 on dist-test-slave-0t40"
server_key: "ea14e8a8a6e72eafb00fa504771a8db3"
server_key_iv: "789790e1d8f7c240c023103570e05ace"
server_key_version: "kuduclusterkey@0"
I20260619 20:38:40.875969 5695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:40.893880 5695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:40.896325 5695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:40.896517 5695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:40.896755 5695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:38:40.897192 5695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:38:40.897281 5695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:40.897377 5695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:38:40.897450 5695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:40.909302 5695 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:40433
I20260619 20:38:40.909883 5695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:38:40.910624 5825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:40433 every 8 connection(s)
I20260619 20:38:40.916319 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 5695
I20260619 20:38:40.916428 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:38:40.916620 25850 external_mini_cluster.cc:1428] Setting key c03ec2828ccd04859a258f2e5d30a799
Jun 19 20:38:40 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901520, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:38:40.923902 5828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:40.911564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42561 (local address 127.25.62.129:36765)
0619 20:38:40.912057 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:40.912102 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:40.912761 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:40.913286 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:40.913291 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:40.913582 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:40.913917 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:40.913923 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:40.915035 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:40.915036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:40.916300 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:40.916304 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:40.916902 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:40.916904 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:40.917097 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:40.917738 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:40.917763 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:40.919583 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:40.921988 (+ 2405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:40.921996 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:40.921998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:40.922320 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:40.922573 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:40.922573 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:40.922574 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:40.922728 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:40.923097 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:40.923103 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:40.923284 (+ 181us) client_negotiation.cc:769] Sending connection context
0619 20:38:40.923412 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 20:38:40.923589 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":103,"threads_started":1}
I20260619 20:38:40.924521 5829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:40.911588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:36765 (local address 127.25.62.190:42561)
0619 20:38:40.912961 (+ 1373us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:40.912964 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:40.912973 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:38:40.913002 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:40.913003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:40.913017 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:40.913125 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:40.914049 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:40.914916 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:40.916416 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:40.916714 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:40.916809 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:40.919709 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:40.919711 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:40.919711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:40.919721 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:40.921880 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:40.922418 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:40.922419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:40.922419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:40.922488 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:40.922831 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:40.922832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:40.922832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:40.922991 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:40.923065 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:40.924303 (+ 1238us) server_negotiation.cc:299] Negotiation successful
0619 20:38:40.924405 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1339,"thread_start_us":98,"threads_started":1}
I20260619 20:38:40.925408 5826 heartbeater.cc:347] Connected to a master server at 127.25.62.190:42561
I20260619 20:38:40.925506 5826 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:40.925697 5826 heartbeater.cc:511] Master 127.25.62.190:42561 requested a full tablet report, sending...
I20260619 20:38:40.926343 5614 ts_manager.cc:194] Registered new tserver with Master: 3812d55bd209489fb934fcf99b7e2685 (127.25.62.129:40433)
WARNING: no policy specified for kudu/127.25.62.130@KRBTEST.COM; defaulting to no policy
I20260619 20:38:40.927258 5614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:36765
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
I20260619 20:38:40.952459 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.62.148:34895/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:42561
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:33621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 20:38:41 dist-test-slave-0t40 krb5kdc[5138](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781901521, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 20:38:41.051234 5834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:41.051497 5834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:41.051534 5834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:41.053323 5834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:41.053381 5834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:41.053421 5834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:38:41.055538 5834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:33621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.25.62.148:34895/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:42561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:41.055785 5834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:41.056020 5834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:41.062664 5839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:41.062975 5834 server_base.cc:1034] running on GCE node
W20260619 20:38:41.062669 5840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:41.063176 5842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:41.063417 5834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:41.063686 5834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:41.064867 5834 hybrid_clock.cc:648] HybridClock initialized: now 1781901521064846 us; error 33 us; skew 500 ppm
Jun 19 20:38:41 dist-test-slave-0t40 krb5kdc[5138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901521, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:41.067981 5834 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:38:41.068449 5834 webserver.cc:460] Webserver started at http://127.25.62.130:39035/ using document root <none> and password file <none>
I20260619 20:38:41.068704 5834 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:41.068758 5834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:41.068907 5834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:38:41 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901521, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
I20260619 20:38:41.087656 5834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "35dcb878dd5c4748a5d594d0bbb835cb"
format_stamp: "Formatted at 2026-06-19 20:38:41 on dist-test-slave-0t40"
server_key: "3b1c4ad96e4380550f72ac15b72b9966"
server_key_iv: "e2562e72c476b99ccc338b2d5339bcc5"
server_key_version: "kuduclusterkey@0"
I20260619 20:38:41.088083 5834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "35dcb878dd5c4748a5d594d0bbb835cb"
format_stamp: "Formatted at 2026-06-19 20:38:41 on dist-test-slave-0t40"
server_key: "3b1c4ad96e4380550f72ac15b72b9966"
server_key_iv: "e2562e72c476b99ccc338b2d5339bcc5"
server_key_version: "kuduclusterkey@0"
I20260619 20:38:41.090703 5834 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 20:38:41.109164 5850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:41.109385 5834 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 20:38:41.109484 5834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "35dcb878dd5c4748a5d594d0bbb835cb"
format_stamp: "Formatted at 2026-06-19 20:38:41 on dist-test-slave-0t40"
server_key: "3b1c4ad96e4380550f72ac15b72b9966"
server_key_iv: "e2562e72c476b99ccc338b2d5339bcc5"
server_key_version: "kuduclusterkey@0"
I20260619 20:38:41.109601 5834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:41.120774 5834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:41.123121 5834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:41.123262 5834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:41.123500 5834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:38:41.123822 5834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:38:41.123881 5834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:41.123919 5834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:38:41.123944 5834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:41.130463 5834 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:41071
I20260619 20:38:41.130947 5834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:38:41.136390 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 5834
I20260619 20:38:41.136482 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:38:41.136674 25850 external_mini_cluster.cc:1428] Setting key 113660f34469aa7f2558863f9d01b34c
I20260619 20:38:41.138357 5963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:41071 every 8 connection(s)
WARNING: no policy specified for kudu/127.25.62.131@KRBTEST.COM; defaulting to no policy
Jun 19 20:38:41 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901521, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.131@KRBTEST.COM" created.
I20260619 20:38:41.151364 5829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:41.138889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:58965 (local address 127.25.62.190:42561)
0619 20:38:41.138940 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:41.138943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:41.139683 (+ 740us) server_negotiation.cc:407] Connection header received
0619 20:38:41.140481 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:41.140483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:41.140499 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:41.140530 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:41.141412 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:41.142237 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:41.143880 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:41.144204 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:41.144250 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:41.146873 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:41.146875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:41.146875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:41.146883 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:41.148424 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:41.149015 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:41.149016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:41.149017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:41.149092 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:41.150512 (+ 1420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:41.150513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:41.150513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:41.150662 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:41.150706 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:41.151097 (+ 391us) server_negotiation.cc:299] Negotiation successful
0619 20:38:41.151198 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:38:41.151888 5966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:41.138572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42561 (local address 127.25.62.130:58965)
0619 20:38:41.139543 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:41.139594 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:41.140363 (+ 769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:41.140707 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:41.140712 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:41.140981 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:41.141291 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:41.141297 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:41.142364 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:41.142365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:41.143756 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:41.143760 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:41.144358 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:41.144360 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:41.144504 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:41.145139 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:41.145152 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:41.146746 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:41.148522 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:41.148529 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:41.148531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:41.148928 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:41.150255 (+ 1327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:41.150257 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:41.150257 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:41.150418 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:41.150755 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:41.150761 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:41.150929 (+ 168us) client_negotiation.cc:769] Sending connection context
0619 20:38:41.151522 (+ 593us) client_negotiation.cc:240] Negotiation successful
0619 20:38:41.151689 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":924,"thread_start_us":111,"threads_started":1}
I20260619 20:38:41.152376 5964 heartbeater.cc:347] Connected to a master server at 127.25.62.190:42561
I20260619 20:38:41.152468 5964 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:41.152650 5964 heartbeater.cc:511] Master 127.25.62.190:42561 requested a full tablet report, sending...
I20260619 20:38:41.153056 5614 ts_manager.cc:194] Registered new tserver with Master: 35dcb878dd5c4748a5d594d0bbb835cb (127.25.62.130:41071)
I20260619 20:38:41.153558 5614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:58965
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
I20260619 20:38:41.164148 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.62.148:34895/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:42561
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:33621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:38:41.271394 5969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:41.271665 5969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:41.271706 5969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:41.273419 5969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:41.273473 5969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:41.273512 5969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:38:41.275559 5969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:33621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.25.62.148:34895/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:42561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:41.275787 5969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:41.276029 5969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:41.283987 5977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:41.284026 5975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:41.284184 5974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:41.284363 5969 server_base.cc:1034] running on GCE node
I20260619 20:38:41.284543 5969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:41.284796 5969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:41.285943 5969 hybrid_clock.cc:648] HybridClock initialized: now 1781901521285911 us; error 47 us; skew 500 ppm
Jun 19 20:38:41 dist-test-slave-0t40 krb5kdc[5138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901521, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:41.288681 5969 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:38:41.289273 5969 webserver.cc:460] Webserver started at http://127.25.62.131:40149/ using document root <none> and password file <none>
I20260619 20:38:41.289539 5969 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:41.289610 5969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:41.289747 5969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 20:38:41 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901521, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
I20260619 20:38:41.304186 5969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "7500717a409e47f6b99442dba14a20c6"
format_stamp: "Formatted at 2026-06-19 20:38:41 on dist-test-slave-0t40"
server_key: "49fc9f41fe6752d1d9661b77fcecb493"
server_key_iv: "2a4ac15634e9c30d9f542f40c9ea635c"
server_key_version: "kuduclusterkey@0"
I20260619 20:38:41.304667 5969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "7500717a409e47f6b99442dba14a20c6"
format_stamp: "Formatted at 2026-06-19 20:38:41 on dist-test-slave-0t40"
server_key: "49fc9f41fe6752d1d9661b77fcecb493"
server_key_iv: "2a4ac15634e9c30d9f542f40c9ea635c"
server_key_version: "kuduclusterkey@0"
I20260619 20:38:41.307188 5969 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260619 20:38:41.332155 5984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:41.332420 5969 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 20:38:41.332515 5969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "7500717a409e47f6b99442dba14a20c6"
format_stamp: "Formatted at 2026-06-19 20:38:41 on dist-test-slave-0t40"
server_key: "49fc9f41fe6752d1d9661b77fcecb493"
server_key_iv: "2a4ac15634e9c30d9f542f40c9ea635c"
server_key_version: "kuduclusterkey@0"
I20260619 20:38:41.332607 5969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:41.373371 5969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:41.375844 5969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:41.376003 5969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:41.376207 5969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:38:41.376549 5969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:38:41.376591 5969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:41.376621 5969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:38:41.376645 5969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:41.384590 5969 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:36273
I20260619 20:38:41.384723 6097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:36273 every 8 connection(s)
I20260619 20:38:41.385018 5969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:38:41.390394 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 5969
I20260619 20:38:41.390499 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:38:41.390709 25850 external_mini_cluster.cc:1428] Setting key 63d6b56bd44d78fbf34c315dd6c69eb9
Jun 19 20:38:41 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901521, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:38:41.397806 5829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:41.386720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:48133 (local address 127.25.62.190:42561)
0619 20:38:41.386780 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:41.386783 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:41.386827 (+ 44us) server_negotiation.cc:407] Connection header received
0619 20:38:41.387581 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:41.387583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:41.387597 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:41.387683 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:41.388382 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:41.389214 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:41.390754 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:41.391073 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:41.391115 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:41.394268 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:41.394269 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:41.394270 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:41.394278 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:41.396129 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:41.396618 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:41.396619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:41.396620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:41.396689 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:41.397018 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:41.397019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:41.397019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:41.397198 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:41.397281 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:41.397607 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 20:38:41.397698 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 20:38:41.397956 6100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:41.386425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42561 (local address 127.25.62.131:48133)
0619 20:38:41.386618 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:41.386666 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:41.387456 (+ 790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:41.387755 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:41.387759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:41.388007 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:41.388272 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:41.388278 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:41.389389 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:41.389390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:41.390650 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:41.390653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:41.391764 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:41.391765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:41.391901 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:41.392478 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:41.392488 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:41.394163 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:41.396230 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:41.396237 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:41.396239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:41.396525 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:41.396772 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:41.396773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:41.396773 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:41.396934 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:41.397283 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:41.397289 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:41.397453 (+ 164us) client_negotiation.cc:769] Sending connection context
0619 20:38:41.397574 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 20:38:41.397723 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":91,"threads_started":1}
I20260619 20:38:41.398362 6098 heartbeater.cc:347] Connected to a master server at 127.25.62.190:42561
I20260619 20:38:41.398452 6098 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:41.398624 6098 heartbeater.cc:511] Master 127.25.62.190:42561 requested a full tablet report, sending...
I20260619 20:38:41.399075 5614 ts_manager.cc:194] Registered new tserver with Master: 7500717a409e47f6b99442dba14a20c6 (127.25.62.131:36273)
I20260619 20:38:41.399557 5614 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:48133
I20260619 20:38:41.402343 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:38:41.403070 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 5695
I20260619 20:38:41.410889 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 5834
I20260619 20:38:41.417286 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 5969
I20260619 20:38:41.423826 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 5582
I20260619 20:38:41.430156 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:42561
--webserver_interface=127.25.62.190
--webserver_port=46155
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:33621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:42561
--ranger_config_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.62.148:34895/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.62.148:34895/kms with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:38:41.512392 6101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:41.512630 6101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:41.512667 6101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:41.513970 6101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:38:41.514039 6101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:41.514063 6101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:41.514096 6101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:38:41.514123 6101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:38:41.515671 6101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:33621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.62.148:34895/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:42561
--ranger_config_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:42561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=46155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:41.515942 6101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:41.516160 6101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:38:41.521090 6101 server_base.cc:1034] running on GCE node
W20260619 20:38:41.521082 6107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:41.521173 6106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:41.521193 6109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:41.521615 6101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:41.521904 6101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:41.523077 6101 hybrid_clock.cc:648] HybridClock initialized: now 1781901521523074 us; error 36 us; skew 500 ppm
Jun 19 20:38:41 dist-test-slave-0t40 krb5kdc[5138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901521, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:41.525625 6101 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:38:41.526089 6101 webserver.cc:460] Webserver started at http://127.25.62.190:46155/ using document root <none> and password file <none>
I20260619 20:38:41.526298 6101 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:41.526371 6101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 20:38:41 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901521, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
I20260619 20:38:41.567065 6101 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:38:41.567900 6117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:41.568104 6101 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:38:41.568215 6101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "16f8263a2d47427fb492bc7865d46f09"
format_stamp: "Formatted at 2026-06-19 20:38:39 on dist-test-slave-0t40"
server_key: "7af728f283da5b46209c2e543924bed4"
server_key_iv: "5f6400f9b5d32f5f7327431b35bac054"
server_key_version: "kuduclusterkey@0"
I20260619 20:38:41.568682 6101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:41.614421 6101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:41.616266 6101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:41.616385 6101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:41.620893 6101 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:42561
I20260619 20:38:41.620934 6169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:42561 every 8 connection(s)
I20260619 20:38:41.621333 6101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:38:41.622519 6170 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:38:41.623232 6170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09: Bootstrap starting.
I20260619 20:38:41.625869 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 6101
I20260619 20:38:41.626154 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.62.148:34895/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:40433
--local_ip_for_outbound_sockets=127.25.62.129
--tserver_master_addrs=127.25.62.190:42561
--webserver_port=34871
--webserver_interface=127.25.62.129
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:33621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.62.148:34895/kms with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:38:41.631470 6170 log.cc:824] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09: Log is configured to *not* fsync() on all Append() calls
I20260619 20:38:41.633611 6170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:38:41.634057 6170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09: Bootstrap complete.
I20260619 20:38:41.638087 6170 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f8263a2d47427fb492bc7865d46f09" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42561 } }
I20260619 20:38:41.638466 6170 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f8263a2d47427fb492bc7865d46f09, State: Initialized, Role: FOLLOWER
I20260619 20:38:41.638679 6170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f8263a2d47427fb492bc7865d46f09" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42561 } }
I20260619 20:38:41.638768 6170 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:38:41.638821 6170 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:38:41.638880 6170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [term 1 FOLLOWER]: Advancing to term 2
I20260619 20:38:41.640417 6170 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f8263a2d47427fb492bc7865d46f09" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42561 } }
I20260619 20:38:41.640537 6170 leader_election.cc:302] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16f8263a2d47427fb492bc7865d46f09; no voters:
I20260619 20:38:41.640707 6170 leader_election.cc:288] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 20:38:41.640815 6174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 20:38:41.640990 6174 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [term 2 LEADER]: Becoming Leader. State: Replica: 16f8263a2d47427fb492bc7865d46f09, State: Running, Role: LEADER
I20260619 20:38:41.640996 6170 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:38:41.642197 6174 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f8263a2d47427fb492bc7865d46f09" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42561 } }
I20260619 20:38:41.643065 6176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "16f8263a2d47427fb492bc7865d46f09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f8263a2d47427fb492bc7865d46f09" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42561 } } }
I20260619 20:38:41.643154 6176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [sys.catalog]: This master's current role is: LEADER
I20260619 20:38:41.643368 6175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16f8263a2d47427fb492bc7865d46f09. Latest consensus state: current_term: 2 leader_uuid: "16f8263a2d47427fb492bc7865d46f09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f8263a2d47427fb492bc7865d46f09" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 42561 } } }
I20260619 20:38:41.643424 6175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09 [sys.catalog]: This master's current role is: LEADER
I20260619 20:38:41.643519 6185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:38:41.643733 6170 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 20:38:41.644012 6185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:38:41.644135 6185 catalog_manager.cc:1259] Loaded cluster ID: f9a06cf59bfe45d3adbf90ac081f6100
I20260619 20:38:41.644176 6185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:38:41.645462 6185 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:38:41.645632 6185 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09: Loaded TSK: 0
I20260619 20:38:41.645844 6185 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 20:38:41.740294 6172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:41.740567 6172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:41.740608 6172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:41.742483 6172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:41.742549 6172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:41.742594 6172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:38:41.744889 6172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:33621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.62.148:34895/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:40433
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=34871
--tserver_master_addrs=127.25.62.190:42561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:41.745157 6172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:41.745410 6172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:41.751802 6207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:41.751881 6172 server_base.cc:1034] running on GCE node
W20260619 20:38:41.752051 6210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:41.752195 6208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:41.752430 6172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:41.752671 6172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:41.753799 6172 hybrid_clock.cc:648] HybridClock initialized: now 1781901521753701 us; error 116 us; skew 500 ppm
Jun 19 20:38:41 dist-test-slave-0t40 krb5kdc[5138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901521, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:41.759404 6172 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:38:41.759984 6172 webserver.cc:460] Webserver started at http://127.25.62.129:34871/ using document root <none> and password file <none>
I20260619 20:38:41.760241 6172 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:41.760334 6172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 20:38:41 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901521, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
I20260619 20:38:41.789490 6172 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.001s sys 0.000s
I20260619 20:38:41.803620 6218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:41.803884 6172 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.001s sys 0.000s
I20260619 20:38:41.804001 6172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "3812d55bd209489fb934fcf99b7e2685"
format_stamp: "Formatted at 2026-06-19 20:38:40 on dist-test-slave-0t40"
server_key: "ea14e8a8a6e72eafb00fa504771a8db3"
server_key_iv: "789790e1d8f7c240c023103570e05ace"
server_key_version: "kuduclusterkey@0"
I20260619 20:38:41.804528 6172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:41.846714 6172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:41.849226 6172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:41.849439 6172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:41.849730 6172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:38:41.850148 6172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:38:41.850241 6172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:41.850344 6172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:38:41.850427 6172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:41.856916 6172 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:40433
I20260619 20:38:41.857466 6172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:38:41.858105 6331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:40433 every 8 connection(s)
I20260619 20:38:41.867429 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 6172
Jun 19 20:38:41 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901521, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:38:41.867743 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.62.148:34895/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:41071
--local_ip_for_outbound_sockets=127.25.62.130
--tserver_master_addrs=127.25.62.190:42561
--webserver_port=39035
--webserver_interface=127.25.62.130
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:33621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.62.148:34895/kms with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:38:41.874265 6334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:41.859120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42561 (local address 127.25.62.129:47873)
0619 20:38:41.859543 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:41.859590 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:41.860243 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:41.861072 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:41.861077 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:41.861306 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:41.861599 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:41.861605 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:41.863417 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:41.863418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:41.864724 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:41.864727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:41.865654 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:41.865656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:41.865789 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:41.866431 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:41.866442 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:41.868775 (+ 2333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:41.872421 (+ 3646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:41.872427 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:41.872429 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:41.872725 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:41.873024 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:41.873025 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:41.873025 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:41.873172 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:41.873632 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:41.873638 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:41.873781 (+ 143us) client_negotiation.cc:769] Sending connection context
0619 20:38:41.873866 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 20:38:41.874035 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":83,"threads_started":1}
I20260619 20:38:41.875349 6335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:41.859183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:47873 (local address 127.25.62.190:42561)
0619 20:38:41.860578 (+ 1395us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:41.860587 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:41.860624 (+ 37us) server_negotiation.cc:407] Connection header received
0619 20:38:41.860768 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:41.860773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:41.860828 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:41.861734 (+ 906us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:41.861841 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:41.863286 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:41.864847 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:41.865531 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:41.867161 (+ 1630us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:41.868914 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:41.868918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:41.868919 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:41.868931 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:41.872310 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:41.872823 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:41.872826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:41.872827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:41.872944 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:41.873256 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:41.873257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:41.873257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:41.873544 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:41.874898 (+ 1354us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:41.874974 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 20:38:41.875106 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1334,"thread_start_us":66,"threads_started":1}
I20260619 20:38:41.876410 6332 heartbeater.cc:347] Connected to a master server at 127.25.62.190:42561
I20260619 20:38:41.876503 6332 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:41.876685 6332 heartbeater.cc:511] Master 127.25.62.190:42561 requested a full tablet report, sending...
W20260619 20:38:41.877602 6332 heartbeater.cc:644] Failed to heartbeat to 127.25.62.190:42561 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 20:38:41.877665 6332 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:41.877722 6332 heartbeater.cc:511] Master 127.25.62.190:42561 requested a full tablet report, sending...
I20260619 20:38:41.878072 6332 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:41.878139 6332 heartbeater.cc:511] Master 127.25.62.190:42561 requested a full tablet report, sending...
W20260619 20:38:41.878449 6332 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 20:38:41.990238 6336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:41.990626 6336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:41.990721 6336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:41.992544 6336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:41.992751 6336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:41.992838 6336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:38:41.995186 6336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:33621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.62.148:34895/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:41071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=39035
--tserver_master_addrs=127.25.62.190:42561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:41.995702 6336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:41.996047 6336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:42.003206 6342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:42.003633 6341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:42.003917 6344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:42.004989 6336 server_base.cc:1034] running on GCE node
I20260619 20:38:42.005348 6336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:42.005666 6336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:42.006839 6336 hybrid_clock.cc:648] HybridClock initialized: now 1781901522006824 us; error 31 us; skew 500 ppm
Jun 19 20:38:42 dist-test-slave-0t40 krb5kdc[5138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:42.009758 6336 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:38:42.010316 6336 webserver.cc:460] Webserver started at http://127.25.62.130:39035/ using document root <none> and password file <none>
I20260619 20:38:42.010589 6336 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:42.010658 6336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 20:38:42 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
I20260619 20:38:42.033814 6336 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260619 20:38:42.037240 6352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:42.037446 6336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 20:38:42.037569 6336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "35dcb878dd5c4748a5d594d0bbb835cb"
format_stamp: "Formatted at 2026-06-19 20:38:41 on dist-test-slave-0t40"
server_key: "3b1c4ad96e4380550f72ac15b72b9966"
server_key_iv: "e2562e72c476b99ccc338b2d5339bcc5"
server_key_version: "kuduclusterkey@0"
I20260619 20:38:42.038096 6336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:42.060709 6336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:42.062963 6336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:42.063143 6336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:42.063431 6336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:38:42.063815 6336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:38:42.063871 6336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:42.063935 6336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:38:42.063966 6336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:42.070293 6336 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:41071
I20260619 20:38:42.070803 6336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:38:42.071992 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 6336
I20260619 20:38:42.072311 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.25.62.148:34895/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:36273
--local_ip_for_outbound_sockets=127.25.62.131
--tserver_master_addrs=127.25.62.190:42561
--webserver_port=40149
--webserver_interface=127.25.62.131
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:33621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.25.62.148:34895/kms with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:38:42.073851 6465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:41071 every 8 connection(s)
Jun 19 20:38:42 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:38:42.095584 6335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:42.074814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:50679 (local address 127.25.62.190:42561)
0619 20:38:42.074904 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:42.074907 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:42.075561 (+ 654us) server_negotiation.cc:407] Connection header received
0619 20:38:42.076201 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:42.076203 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:42.076217 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:42.076279 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:42.076826 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:42.077583 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:42.082727 (+ 5144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:42.083081 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:42.083133 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:42.088200 (+ 5067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:42.088202 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:42.088202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:42.088211 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:42.090076 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:42.090574 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:42.090575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:42.090575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:42.090628 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:42.090880 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:42.090881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:42.090881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:42.091010 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:42.091049 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:42.095374 (+ 4325us) server_negotiation.cc:299] Negotiation successful
0619 20:38:42.095479 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 20:38:42.096153 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:42.074528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42561 (local address 127.25.62.130:50679)
0619 20:38:42.075437 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:42.075472 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:42.076104 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:42.076352 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:42.076356 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:42.076525 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:42.076720 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:42.076724 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:42.081141 (+ 4417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:42.081143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:42.082608 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:42.082611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:42.083272 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:42.083274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:42.084238 (+ 964us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:42.084928 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:42.084947 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:42.087187 (+ 2240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:42.090174 (+ 2987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:42.090180 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:42.090183 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:42.090487 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:42.090694 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:42.090695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:42.090695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:42.090809 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:42.091091 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:42.091095 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:42.091249 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 20:38:42.095700 (+ 4451us) client_negotiation.cc:240] Negotiation successful
0619 20:38:42.095885 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"thread_start_us":82,"threads_started":1}
I20260619 20:38:42.096637 6466 heartbeater.cc:347] Connected to a master server at 127.25.62.190:42561
I20260619 20:38:42.096735 6466 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:42.096940 6466 heartbeater.cc:511] Master 127.25.62.190:42561 requested a full tablet report, sending...
W20260619 20:38:42.097677 6466 heartbeater.cc:644] Failed to heartbeat to 127.25.62.190:42561 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 20:38:42.097774 6466 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:42.097851 6466 heartbeater.cc:511] Master 127.25.62.190:42561 requested a full tablet report, sending...
I20260619 20:38:42.098188 6466 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:42.098258 6466 heartbeater.cc:511] Master 127.25.62.190:42561 requested a full tablet report, sending...
W20260619 20:38:42.098554 6466 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 20:38:42.196769 6468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:42.197162 6468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:42.197257 6468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:42.199057 6468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:42.199173 6468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:42.199263 6468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:38:42.201473 6468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:33621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.25.62.148:34895/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:36273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=40149
--tserver_master_addrs=127.25.62.190:42561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:42.202113 6468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:42.202440 6468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:42.209345 6475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:42.209368 6476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:42.209549 6468 server_base.cc:1034] running on GCE node
W20260619 20:38:42.209926 6478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:42.210366 6468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:42.210615 6468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:42.211869 6468 hybrid_clock.cc:648] HybridClock initialized: now 1781901522211848 us; error 34 us; skew 500 ppm
Jun 19 20:38:42 dist-test-slave-0t40 krb5kdc[5138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:42.216040 6468 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:38:42.216555 6468 webserver.cc:460] Webserver started at http://127.25.62.131:40149/ using document root <none> and password file <none>
I20260619 20:38:42.216825 6468 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:42.216892 6468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 20:38:42 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
I20260619 20:38:42.237788 6468 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260619 20:38:42.238818 6486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:42.239029 6468 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:38:42.239105 6468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "7500717a409e47f6b99442dba14a20c6"
format_stamp: "Formatted at 2026-06-19 20:38:41 on dist-test-slave-0t40"
server_key: "49fc9f41fe6752d1d9661b77fcecb493"
server_key_iv: "2a4ac15634e9c30d9f542f40c9ea635c"
server_key_version: "kuduclusterkey@0"
I20260619 20:38:42.239549 6468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:42.258428 6468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:42.260865 6468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:42.261111 6468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:42.261379 6468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:38:42.261778 6468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:38:42.261873 6468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:42.261940 6468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:38:42.261994 6468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:42.268514 6468 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:36273
I20260619 20:38:42.268621 6599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:36273 every 8 connection(s)
I20260619 20:38:42.269099 6468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:38:42.271927 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 6468
Jun 19 20:38:42 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:38:42.281534 6602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:42.270164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:42561 (local address 127.25.62.131:32873)
0619 20:38:42.270535 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:42.270579 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:42.271168 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:42.271480 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:42.271484 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:42.271700 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:42.272119 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:42.272126 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:42.273172 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:42.273187 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:42.274452 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:42.274455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:42.275044 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:42.275045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:42.275185 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:42.275786 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:42.275797 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:42.277598 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:42.279447 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:42.279453 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:42.279455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:42.279744 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:42.280356 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:42.280357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:42.280357 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:42.280503 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:42.280802 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:42.280808 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:42.280969 (+ 161us) client_negotiation.cc:769] Sending connection context
0619 20:38:42.281137 (+ 168us) client_negotiation.cc:240] Negotiation successful
0619 20:38:42.281306 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":73,"threads_started":1}
I20260619 20:38:42.282096 6335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:42.270420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:32873 (local address 127.25.62.190:42561)
0619 20:38:42.270463 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:42.270466 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:42.270688 (+ 222us) server_negotiation.cc:407] Connection header received
0619 20:38:42.271301 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:42.271303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:42.271315 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:42.271407 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:42.272229 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:42.272992 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:42.274591 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:42.274899 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:42.274965 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:42.277705 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:42.277706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:42.277707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:42.277713 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:42.279351 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:42.279833 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:42.279835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:42.279835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:42.279895 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:42.280587 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:42.280588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:42.280589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:42.280719 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:42.280777 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:42.281922 (+ 1145us) server_negotiation.cc:299] Negotiation successful
0619 20:38:42.282007 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:38:42.282809 6600 heartbeater.cc:347] Connected to a master server at 127.25.62.190:42561
I20260619 20:38:42.282902 6600 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:42.283078 6600 heartbeater.cc:511] Master 127.25.62.190:42561 requested a full tablet report, sending...
W20260619 20:38:42.283722 6600 heartbeater.cc:644] Failed to heartbeat to 127.25.62.190:42561 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 20:38:42.283797 6600 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:42.283859 6600 heartbeater.cc:511] Master 127.25.62.190:42561 requested a full tablet report, sending...
I20260619 20:38:42.284209 6600 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:42.284294 6600 heartbeater.cc:511] Master 127.25.62.190:42561 requested a full tablet report, sending...
W20260619 20:38:42.284621 6600 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 20:38:42.292271 6604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:42.272299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43656 (local address 127.25.62.190:42561)
0619 20:38:42.272581 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:42.272583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:42.272589 (+ 6us) server_negotiation.cc:407] Connection header received
0619 20:38:42.272608 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:42.272609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:42.272620 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:42.272671 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:42.273510 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:42.274216 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:42.275287 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:42.275581 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:42.275634 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:42.289109 (+ 13475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:42.289111 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:42.289111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:42.289118 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:42.291023 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:42.291444 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:42.291445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:42.291446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:42.291505 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:42.291720 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:42.291720 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:42.291721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:42.291858 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:42.292014 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:42.292093 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 20:38:42.292173 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":65,"threads_started":1}
Jun 19 20:38:42 dist-test-slave-0t40 krb5kdc[5138](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781901522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:42.878610 6332 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:42.878758 6332 heartbeater.cc:511] Master 127.25.62.190:42561 requested a full tablet report, sending...
I20260619 20:38:42.879163 6134 ts_manager.cc:194] Registered new tserver with Master: 3812d55bd209489fb934fcf99b7e2685 (127.25.62.129:40433)
I20260619 20:38:42.880160 6134 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:47873
I20260619 20:38:43.098721 6466 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:43.098922 6466 heartbeater.cc:511] Master 127.25.62.190:42561 requested a full tablet report, sending...
I20260619 20:38:43.099323 6134 ts_manager.cc:194] Registered new tserver with Master: 35dcb878dd5c4748a5d594d0bbb835cb (127.25.62.130:41071)
I20260619 20:38:43.099748 6134 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:50679
Jun 19 20:38:43 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
I20260619 20:38:43.284781 6600 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:43.284942 6600 heartbeater.cc:511] Master 127.25.62.190:42561 requested a full tablet report, sending...
I20260619 20:38:43.285262 6134 ts_manager.cc:194] Registered new tserver with Master: 7500717a409e47f6b99442dba14a20c6 (127.25.62.131:36273)
I20260619 20:38:43.285658 6134 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:32873
I20260619 20:38:43.289342 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:38:43.296622 6621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:43.290264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58492 (local address 127.25.62.190:42561)
0619 20:38:43.290410 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:43.290413 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:43.290492 (+ 79us) server_negotiation.cc:407] Connection header received
0619 20:38:43.290559 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:43.290559 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:43.290571 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:43.290632 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:43.291172 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.291670 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.292379 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.292592 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.292667 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:43.293935 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:43.293937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:43.293937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:43.293944 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:43.295484 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:43.295844 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:43.295845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:43.295845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:43.295906 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:43.296104 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:43.296105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:43.296105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:43.296214 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:43.296315 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:43.296422 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 20:38:43.296503 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260619 20:38:43.298650 6134 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:38:43.300252 6134 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:38:43.308544 6632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:43.303254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55120 (local address 127.25.62.129:40433)
0619 20:38:43.303384 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:43.303388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:43.305253 (+ 1865us) server_negotiation.cc:407] Connection header received
0619 20:38:43.305402 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:43.305405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:43.305423 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:43.305496 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:38:43.305769 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.306437 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.307413 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.308085 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.308179 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:43.308237 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:43.308394 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 20:38:43.308468 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":84,"threads_started":1}
I20260619 20:38:43.308542 6630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:43.302831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:40433 (local address 127.0.0.1:55120)
0619 20:38:43.305152 (+ 2321us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:43.305183 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:43.305266 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:43.305628 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:43.305631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:43.305633 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:38:43.305694 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:43.305695 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.306519 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.306520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:43.307309 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:43.307310 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.308182 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.308183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:43.308264 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:43.308277 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 20:38:43.308377 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 20:38:43.308466 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2287,"thread_start_us":82,"threads_started":1}
I20260619 20:38:43.309546 6258 tablet_service.cc:1467] Processing CreateTablet for tablet 2adbac84bae14a52a9455bd5a6e82f02 (DEFAULT_TABLE table=test-table [id=ea2c0f0b0d0a470fadb6a3b47b6f5d00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:38:43.309913 6258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2adbac84bae14a52a9455bd5a6e82f02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:38:43.312930 6629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:43.302959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47560 (local address 127.25.62.130:41071)
0619 20:38:43.303105 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:43.303110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:43.308772 (+ 5662us) server_negotiation.cc:407] Connection header received
0619 20:38:43.308875 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:43.308878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:43.308894 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:43.308968 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:38:43.309227 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.310220 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.311715 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.312478 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.312533 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:43.312588 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:43.312799 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 20:38:43.312843 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
I20260619 20:38:43.313186 6630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:43.302839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:41071 (local address 127.0.0.1:47560)
0619 20:38:43.308722 (+ 5883us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:43.308731 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:43.308799 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:43.309120 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:43.309121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:43.309122 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:38:43.309150 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:43.309151 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.310351 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.310352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:43.311611 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:43.311614 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.312649 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.312650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:43.312709 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:43.312712 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:38:43.313076 (+ 364us) client_negotiation.cc:240] Negotiation successful
0619 20:38:43.313118 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5872,"thread_start_us":80,"threads_started":1}
I20260619 20:38:43.313874 6392 tablet_service.cc:1467] Processing CreateTablet for tablet 2adbac84bae14a52a9455bd5a6e82f02 (DEFAULT_TABLE table=test-table [id=ea2c0f0b0d0a470fadb6a3b47b6f5d00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:38:43.314203 6392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2adbac84bae14a52a9455bd5a6e82f02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:38:43.317432 6630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:43.303056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:36273 (local address 127.0.0.1:44010)
0619 20:38:43.313257 (+ 10201us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:43.313269 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:43.313317 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:43.313608 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:43.313609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:43.313610 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:38:43.313638 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:43.313639 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.315141 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.315142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:43.316176 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:43.316179 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.317196 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.317198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:43.317260 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:43.317263 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:38:43.317316 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:38:43.317366 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10189,"thread_start_us":65,"threads_started":1}
I20260619 20:38:43.317711 6631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:43.303168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44010 (local address 127.25.62.131:36273)
0619 20:38:43.309126 (+ 5958us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:43.309131 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:43.313396 (+ 4265us) server_negotiation.cc:407] Connection header received
0619 20:38:43.313435 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:43.313437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:43.313453 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:43.313522 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:38:43.313709 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.315043 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.316276 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.317008 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.317106 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:43.317161 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:43.317582 (+ 421us) server_negotiation.cc:299] Negotiation successful
0619 20:38:43.317629 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5919,"thread_start_us":61,"threads_started":1}
I20260619 20:38:43.318396 6523 tablet_service.cc:1467] Processing CreateTablet for tablet 2adbac84bae14a52a9455bd5a6e82f02 (DEFAULT_TABLE table=test-table [id=ea2c0f0b0d0a470fadb6a3b47b6f5d00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:38:43.318549 6634 tablet_bootstrap.cc:492] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685: Bootstrap starting.
I20260619 20:38:43.318778 6523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2adbac84bae14a52a9455bd5a6e82f02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:38:43.318836 6633 tablet_bootstrap.cc:492] T 2adbac84bae14a52a9455bd5a6e82f02 P 35dcb878dd5c4748a5d594d0bbb835cb: Bootstrap starting.
I20260619 20:38:43.319348 6634 tablet_bootstrap.cc:654] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685: Neither blocks nor log segments found. Creating new log.
I20260619 20:38:43.319386 6633 tablet_bootstrap.cc:654] T 2adbac84bae14a52a9455bd5a6e82f02 P 35dcb878dd5c4748a5d594d0bbb835cb: Neither blocks nor log segments found. Creating new log.
I20260619 20:38:43.319680 6633 log.cc:824] T 2adbac84bae14a52a9455bd5a6e82f02 P 35dcb878dd5c4748a5d594d0bbb835cb: Log is configured to *not* fsync() on all Append() calls
I20260619 20:38:43.319716 6634 log.cc:824] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685: Log is configured to *not* fsync() on all Append() calls
I20260619 20:38:43.322446 6637 tablet_bootstrap.cc:492] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6: Bootstrap starting.
I20260619 20:38:43.323160 6637 tablet_bootstrap.cc:654] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6: Neither blocks nor log segments found. Creating new log.
I20260619 20:38:43.324070 6637 log.cc:824] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6: Log is configured to *not* fsync() on all Append() calls
I20260619 20:38:43.324898 6637 tablet_bootstrap.cc:492] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6: No bootstrap required, opened a new log
I20260619 20:38:43.324965 6637 ts_tablet_manager.cc:1397] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:38:43.325950 6634 tablet_bootstrap.cc:492] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685: No bootstrap required, opened a new log
I20260619 20:38:43.326022 6634 ts_tablet_manager.cc:1397] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260619 20:38:43.326998 6637 raft_consensus.cc:348] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3812d55bd209489fb934fcf99b7e2685" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40433 } } peers { permanent_uuid: "7500717a409e47f6b99442dba14a20c6" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36273 } } peers { permanent_uuid: "35dcb878dd5c4748a5d594d0bbb835cb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 41071 } }
I20260619 20:38:43.327149 6637 raft_consensus.cc:374] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:38:43.327181 6637 raft_consensus.cc:729] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7500717a409e47f6b99442dba14a20c6, State: Initialized, Role: FOLLOWER
I20260619 20:38:43.327316 6637 consensus_queue.cc:260] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3812d55bd209489fb934fcf99b7e2685" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40433 } } peers { permanent_uuid: "7500717a409e47f6b99442dba14a20c6" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36273 } } peers { permanent_uuid: "35dcb878dd5c4748a5d594d0bbb835cb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 41071 } }
I20260619 20:38:43.327576 6637 ts_tablet_manager.cc:1428] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:38:43.327987 6600 heartbeater.cc:503] Master 127.25.62.190:42561 was elected leader, sending a full tablet report...
I20260619 20:38:43.328428 6634 raft_consensus.cc:348] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3812d55bd209489fb934fcf99b7e2685" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40433 } } peers { permanent_uuid: "7500717a409e47f6b99442dba14a20c6" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36273 } } peers { permanent_uuid: "35dcb878dd5c4748a5d594d0bbb835cb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 41071 } }
I20260619 20:38:43.328588 6634 raft_consensus.cc:374] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:38:43.328626 6634 raft_consensus.cc:729] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3812d55bd209489fb934fcf99b7e2685, State: Initialized, Role: FOLLOWER
I20260619 20:38:43.329128 6633 tablet_bootstrap.cc:492] T 2adbac84bae14a52a9455bd5a6e82f02 P 35dcb878dd5c4748a5d594d0bbb835cb: No bootstrap required, opened a new log
I20260619 20:38:43.329213 6633 ts_tablet_manager.cc:1397] T 2adbac84bae14a52a9455bd5a6e82f02 P 35dcb878dd5c4748a5d594d0bbb835cb: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260619 20:38:43.331902 6633 raft_consensus.cc:348] T 2adbac84bae14a52a9455bd5a6e82f02 P 35dcb878dd5c4748a5d594d0bbb835cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3812d55bd209489fb934fcf99b7e2685" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40433 } } peers { permanent_uuid: "7500717a409e47f6b99442dba14a20c6" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36273 } } peers { permanent_uuid: "35dcb878dd5c4748a5d594d0bbb835cb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 41071 } }
I20260619 20:38:43.332077 6633 raft_consensus.cc:374] T 2adbac84bae14a52a9455bd5a6e82f02 P 35dcb878dd5c4748a5d594d0bbb835cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:38:43.332115 6633 raft_consensus.cc:729] T 2adbac84bae14a52a9455bd5a6e82f02 P 35dcb878dd5c4748a5d594d0bbb835cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35dcb878dd5c4748a5d594d0bbb835cb, State: Initialized, Role: FOLLOWER
I20260619 20:38:43.332257 6634 consensus_queue.cc:260] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3812d55bd209489fb934fcf99b7e2685" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40433 } } peers { permanent_uuid: "7500717a409e47f6b99442dba14a20c6" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36273 } } peers { permanent_uuid: "35dcb878dd5c4748a5d594d0bbb835cb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 41071 } }
I20260619 20:38:43.332347 6633 consensus_queue.cc:260] T 2adbac84bae14a52a9455bd5a6e82f02 P 35dcb878dd5c4748a5d594d0bbb835cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3812d55bd209489fb934fcf99b7e2685" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40433 } } peers { permanent_uuid: "7500717a409e47f6b99442dba14a20c6" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36273 } } peers { permanent_uuid: "35dcb878dd5c4748a5d594d0bbb835cb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 41071 } }
I20260619 20:38:43.332542 6634 ts_tablet_manager.cc:1428] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260619 20:38:43.332653 6332 heartbeater.cc:503] Master 127.25.62.190:42561 was elected leader, sending a full tablet report...
I20260619 20:38:43.333353 6633 ts_tablet_manager.cc:1428] T 2adbac84bae14a52a9455bd5a6e82f02 P 35dcb878dd5c4748a5d594d0bbb835cb: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260619 20:38:43.333530 6466 heartbeater.cc:503] Master 127.25.62.190:42561 was elected leader, sending a full tablet report...
W20260619 20:38:43.360404 6333 tablet.cc:2369] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 20:38:43 dist-test-slave-0t40 krb5kdc[5138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901522, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for HTTP/127.25.62.148@KRBTEST.COM
I20260619 20:38:43.448487 6640 raft_consensus.cc:482] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:38:43.448633 6640 raft_consensus.cc:504] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3812d55bd209489fb934fcf99b7e2685" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40433 } } peers { permanent_uuid: "7500717a409e47f6b99442dba14a20c6" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36273 } } peers { permanent_uuid: "35dcb878dd5c4748a5d594d0bbb835cb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 41071 } }
I20260619 20:38:43.449014 6640 leader_election.cc:288] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7500717a409e47f6b99442dba14a20c6 (127.25.62.131:36273), 35dcb878dd5c4748a5d594d0bbb835cb (127.25.62.130:41071)
I20260619 20:38:43.454596 6629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:43.449514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:34443 (local address 127.25.62.130:41071)
0619 20:38:43.449641 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:43.449643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:43.449653 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:38:43.449683 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:43.449684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:43.449694 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:43.449766 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:38:43.451655 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.452299 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.453612 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.454141 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.454195 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:43.454204 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:43.454461 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 20:38:43.454521 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260619 20:38:43.454972 6643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:43.449209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:41071 (local address 127.25.62.129:34443)
0619 20:38:43.449407 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:43.449419 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:43.449476 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:43.451428 (+ 1952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:43.451429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:43.451431 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:38:43.451535 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:43.451537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.452415 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.452416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:43.453500 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:43.453503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.454294 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.454295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:43.454365 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:43.454369 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:38:43.454831 (+ 462us) client_negotiation.cc:240] Negotiation successful
0619 20:38:43.454876 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":139,"threads_started":1}
I20260619 20:38:43.455319 6420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2adbac84bae14a52a9455bd5a6e82f02" candidate_uuid: "3812d55bd209489fb934fcf99b7e2685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dcb878dd5c4748a5d594d0bbb835cb" is_pre_election: true
I20260619 20:38:43.455502 6420 raft_consensus.cc:2454] T 2adbac84bae14a52a9455bd5a6e82f02 P 35dcb878dd5c4748a5d594d0bbb835cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3812d55bd209489fb934fcf99b7e2685 in term 0.
I20260619 20:38:43.455765 6221 leader_election.cc:302] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35dcb878dd5c4748a5d594d0bbb835cb, 3812d55bd209489fb934fcf99b7e2685; no voters:
I20260619 20:38:43.455915 6640 raft_consensus.cc:2781] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:38:43.455967 6640 raft_consensus.cc:482] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:38:43.455999 6640 raft_consensus.cc:3037] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:38:43.456979 6640 raft_consensus.cc:504] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3812d55bd209489fb934fcf99b7e2685" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40433 } } peers { permanent_uuid: "7500717a409e47f6b99442dba14a20c6" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36273 } } peers { permanent_uuid: "35dcb878dd5c4748a5d594d0bbb835cb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 41071 } }
I20260619 20:38:43.457154 6640 leader_election.cc:288] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 [CANDIDATE]: Term 1 election: Requested vote from peers 7500717a409e47f6b99442dba14a20c6 (127.25.62.131:36273), 35dcb878dd5c4748a5d594d0bbb835cb (127.25.62.130:41071)
I20260619 20:38:43.457522 6420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2adbac84bae14a52a9455bd5a6e82f02" candidate_uuid: "3812d55bd209489fb934fcf99b7e2685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35dcb878dd5c4748a5d594d0bbb835cb"
I20260619 20:38:43.457608 6420 raft_consensus.cc:3037] T 2adbac84bae14a52a9455bd5a6e82f02 P 35dcb878dd5c4748a5d594d0bbb835cb [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:38:43.458582 6420 raft_consensus.cc:2454] T 2adbac84bae14a52a9455bd5a6e82f02 P 35dcb878dd5c4748a5d594d0bbb835cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3812d55bd209489fb934fcf99b7e2685 in term 1.
I20260619 20:38:43.458827 6631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:43.449583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:60971 (local address 127.25.62.131:36273)
0619 20:38:43.449649 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:43.449651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:43.449977 (+ 326us) server_negotiation.cc:407] Connection header received
0619 20:38:43.450080 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:43.450081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:43.450089 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:43.450116 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:38:43.450319 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.451028 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.456491 (+ 5463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.457004 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.457068 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:43.457076 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:43.458707 (+ 1631us) server_negotiation.cc:299] Negotiation successful
0619 20:38:43.458760 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:38:43.458849 6221 leader_election.cc:302] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35dcb878dd5c4748a5d594d0bbb835cb, 3812d55bd209489fb934fcf99b7e2685; no voters:
I20260619 20:38:43.458946 6640 raft_consensus.cc:2781] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:38:43.459112 6640 raft_consensus.cc:686] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 [term 1 LEADER]: Becoming Leader. State: Replica: 3812d55bd209489fb934fcf99b7e2685, State: Running, Role: LEADER
I20260619 20:38:43.459219 6640 consensus_queue.cc:237] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3812d55bd209489fb934fcf99b7e2685" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40433 } } peers { permanent_uuid: "7500717a409e47f6b99442dba14a20c6" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36273 } } peers { permanent_uuid: "35dcb878dd5c4748a5d594d0bbb835cb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 41071 } }
I20260619 20:38:43.459401 6644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:43.449397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:36273 (local address 127.25.62.129:60971)
0619 20:38:43.449797 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:43.449807 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:43.450005 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:43.450173 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:43.450174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:43.450175 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:38:43.450253 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:43.450255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.451687 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.451688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:43.455782 (+ 4094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:43.455785 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.458538 (+ 2753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.458539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:43.458612 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:43.458615 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:38:43.459223 (+ 608us) client_negotiation.cc:240] Negotiation successful
0619 20:38:43.459303 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":76,"threads_started":1}
I20260619 20:38:43.459812 6546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2adbac84bae14a52a9455bd5a6e82f02" candidate_uuid: "3812d55bd209489fb934fcf99b7e2685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7500717a409e47f6b99442dba14a20c6" is_pre_election: true
I20260619 20:38:43.459975 6546 raft_consensus.cc:2454] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3812d55bd209489fb934fcf99b7e2685 in term 0.
I20260619 20:38:43.460203 6548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2adbac84bae14a52a9455bd5a6e82f02" candidate_uuid: "3812d55bd209489fb934fcf99b7e2685" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7500717a409e47f6b99442dba14a20c6"
I20260619 20:38:43.460281 6548 raft_consensus.cc:3037] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:38:43.461261 6548 raft_consensus.cc:2454] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3812d55bd209489fb934fcf99b7e2685 in term 1.
I20260619 20:38:43.462078 6134 catalog_manager.cc:5495] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3812d55bd209489fb934fcf99b7e2685 (127.25.62.129). New cstate: current_term: 1 leader_uuid: "3812d55bd209489fb934fcf99b7e2685" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3812d55bd209489fb934fcf99b7e2685" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 40433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7500717a409e47f6b99442dba14a20c6" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35dcb878dd5c4748a5d594d0bbb835cb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 41071 } health_report { overall_health: UNKNOWN } } }
I20260619 20:38:43.489929 6632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:43.486337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55134 (local address 127.25.62.129:40433)
0619 20:38:43.486429 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:43.486431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:43.486438 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:38:43.486478 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:43.486479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:43.486500 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:43.486575 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:38:43.486781 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.487509 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.488645 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:43.488937 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:43.489010 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:43.489283 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:38:43.489445 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:38:43.489533 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:43.489814 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 20:38:43.489840 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 20:38:43.493681 6420 raft_consensus.cc:1261] T 2adbac84bae14a52a9455bd5a6e82f02 P 35dcb878dd5c4748a5d594d0bbb835cb [term 1 FOLLOWER]: Refusing update from remote peer 3812d55bd209489fb934fcf99b7e2685: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:38:43.494109 6640 consensus_queue.cc:1035] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35dcb878dd5c4748a5d594d0bbb835cb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 41071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:38:43.496577 6548 raft_consensus.cc:1261] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6 [term 1 FOLLOWER]: Refusing update from remote peer 3812d55bd209489fb934fcf99b7e2685: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:38:43.497004 6640 consensus_queue.cc:1035] T 2adbac84bae14a52a9455bd5a6e82f02 P 3812d55bd209489fb934fcf99b7e2685 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7500717a409e47f6b99442dba14a20c6" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 36273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:38:43.499694 6649 mvcc.cc:205] Tried to move back new op lower bound from 7298668640226066432 to 7298668640089640960. Current Snapshot: MvccSnapshot[applied={T|T < 7298668640226066432}]
I20260619 20:38:43.499889 6650 mvcc.cc:205] Tried to move back new op lower bound from 7298668640226066432 to 7298668640089640960. Current Snapshot: MvccSnapshot[applied={T|T < 7298668640226066432 or (T in {7298668640226066432})}]
I20260619 20:38:43.503000 6134 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:38:43.503108 6134 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:38:43.504027 6134 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 16f8263a2d47427fb492bc7865d46f09: Sending DeleteTablet for 3 replicas of tablet 2adbac84bae14a52a9455bd5a6e82f02
I20260619 20:38:43.504521 6258 tablet_service.cc:1514] Processing DeleteTablet for tablet 2adbac84bae14a52a9455bd5a6e82f02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:38:43 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:55120
I20260619 20:38:43.504537 6392 tablet_service.cc:1514] Processing DeleteTablet for tablet 2adbac84bae14a52a9455bd5a6e82f02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:38:43 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:47560
I20260619 20:38:43.504742 6523 tablet_service.cc:1514] Processing DeleteTablet for tablet 2adbac84bae14a52a9455bd5a6e82f02 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:38:43 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:44010
I20260619 20:38:43.505008 6659 tablet_replica.cc:330] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6: stopping tablet replica
I20260619 20:38:43.505012 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 6172
I20260619 20:38:43.505195 6659 raft_consensus.cc:2229] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:38:43.507064 6659 pending_rounds.cc:62] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6: Trying to abort 2 pending ops.
I20260619 20:38:43.507189 6659 pending_rounds.cc:68] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7298668640226066432 op_type: WRITE_OP write_request { tablet_id: "2adbac84bae14a52a9455bd5a6e82f02" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\377\323\326\321\006\"8\n\ntest-admin\022*\n ea2c0f0b0d0a470fadb6a3b47b6f5d00\020\001\030\001 \001(\001" signature: "<redacted>""s\217l\355\2779\0309\351KY!\302\026\245\323\302\270l_\222\016\214\347\372(\225R\206\\U~\177\227\023}\334\n\017`#\202P\034l4<\020S2Dn\201\211\257q\342\262\341\310\343\320\010h" signing_key_seq_num: 0 } } request_id { client_id: "0354f616826b444ab6924bef449c8e0c" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260619 20:38:43.507553 6659 pending_rounds.cc:68] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298668640089640960 op_type: NO_OP noop_request { }
I20260619 20:38:43.507670 6659 raft_consensus.cc:2866] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260619 20:38:43.507740 6659 raft_consensus.cc:2258] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:38:43.507987 6658 tablet_replica.cc:330] T 2adbac84bae14a52a9455bd5a6e82f02 P 35dcb878dd5c4748a5d594d0bbb835cb: stopping tablet replica
I20260619 20:38:43.508152 6658 raft_consensus.cc:2229] T 2adbac84bae14a52a9455bd5a6e82f02 P 35dcb878dd5c4748a5d594d0bbb835cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:38:43.508356 6658 raft_consensus.cc:2258] T 2adbac84bae14a52a9455bd5a6e82f02 P 35dcb878dd5c4748a5d594d0bbb835cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:38:43.509735 6658 ts_tablet_manager.cc:1905] T 2adbac84bae14a52a9455bd5a6e82f02 P 35dcb878dd5c4748a5d594d0bbb835cb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 20:38:43.514858 6489 connection.cc:535] server connection from 127.25.62.129:60971 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 20:38:43.515405 6121 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.62.129:40433 (error 108)
I20260619 20:38:43.516213 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 6336
I20260619 20:38:43.516613 6659 ts_tablet_manager.cc:1905] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:38:43.518415 6659 ts_tablet_manager.cc:1918] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:38:43.518666 6659 log.cc:1196] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/2adbac84bae14a52a9455bd5a6e82f02
I20260619 20:38:43.518991 6659 ts_tablet_manager.cc:1939] T 2adbac84bae14a52a9455bd5a6e82f02 P 7500717a409e47f6b99442dba14a20c6: Deleting consensus metadata
I20260619 20:38:43.521916 6121 catalog_manager.cc:4841] TS 7500717a409e47f6b99442dba14a20c6 (127.25.62.131:36273): tablet 2adbac84bae14a52a9455bd5a6e82f02 (table test-table [id=ea2c0f0b0d0a470fadb6a3b47b6f5d00]) successfully deleted
I20260619 20:38:43.525585 6630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:43.525428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:40433 (local address 127.0.0.1:55148)
0619 20:38:43.525493 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:43.525543 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.129:40433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 20:38:43.525832 6121 catalog_manager.cc:4584] TS 3812d55bd209489fb934fcf99b7e2685 (127.25.62.129:40433): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2adbac84bae14a52a9455bd5a6e82f02: Network error: Client connection negotiation failed: client connection to 127.25.62.129:40433: connect: Connection refused (error 111)
I20260619 20:38:43.531332 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 6468
I20260619 20:38:43.534258 6630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:43.534131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:41071 (local address 127.0.0.1:47576)
0619 20:38:43.534185 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:43.534214 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.130:41071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 20:38:43.536801 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 6101
I20260619 20:38:43.543701 25850 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 20:38:43.887821 25850 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 20:38:43.887967 25850 mini_ranger.cc:66] Stopping Ranger...
I20260619 20:38:44.232157 25850 mini_ranger.cc:68] Stopped Ranger
2026-06-19 20:38:44.232 UTC [5168] LOG: received smart shutdown request
2026-06-19 20:38:44.235 UTC [5168] LOG: background worker "logical replication launcher" (PID 5177) exited with exit code 1
2026-06-19 20:38:44.235 UTC [5172] LOG: shutting down
2026-06-19 20:38:44.399 UTC [5168] LOG: database system is shut down
2026-06-19T20:38:44Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (46903 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:38:44 dist-test-slave-0t40 krb5kdc[6669](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:38:44 dist-test-slave-0t40 krb5kdc[6669](info): set up 2 sockets
Jun 19 20:38:44 dist-test-slave-0t40 krb5kdc[6669](info): commencing operation
krb5kdc: starting...
W20260619 20:38:45.545300 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:38:45 dist-test-slave-0t40 krb5kdc[6669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:38:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:38:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:38:45.663427 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:46257
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:39463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:46257,127.25.62.189:40557,127.25.62.188:46195
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:38:45.742340 6682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:45.742566 6682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:45.742591 6682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:45.743721 6682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:38:45.743759 6682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:45.743774 6682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:45.743788 6682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:38:45.743799 6682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:38:45.745378 6682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:39463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:46257,127.25.62.189:40557,127.25.62.188:46195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:46257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:45.745626 6682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:45.745846 6682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:45.750612 6687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:45.750617 6690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:45.750615 6688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:45.750902 6682 server_base.cc:1034] running on GCE node
I20260619 20:38:45.751156 6682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:45.751441 6682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:45.752581 6682 hybrid_clock.cc:648] HybridClock initialized: now 1781901525752566 us; error 40 us; skew 500 ppm
Jun 19 20:38:45 dist-test-slave-0t40 krb5kdc[6669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901525, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:45.760663 6682 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:38:45.761090 6682 webserver.cc:460] Webserver started at http://127.25.62.190:39719/ using document root <none> and password file <none>
I20260619 20:38:45.761320 6682 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:45.761370 6682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:45.761509 6682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:38:45.762567 6682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "bc42f822d59749eb8ed64ddeab877a47"
format_stamp: "Formatted at 2026-06-19 20:38:45 on dist-test-slave-0t40"
server_key: "3ee307f67c9699a3c0d2c01cb2a38224"
server_key_iv: "7b657a0cfaf8dff5e2b425bb19168ff0"
server_key_version: "encryptionkey@0"
I20260619 20:38:45.762918 6682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "bc42f822d59749eb8ed64ddeab877a47"
format_stamp: "Formatted at 2026-06-19 20:38:45 on dist-test-slave-0t40"
server_key: "3ee307f67c9699a3c0d2c01cb2a38224"
server_key_iv: "7b657a0cfaf8dff5e2b425bb19168ff0"
server_key_version: "encryptionkey@0"
I20260619 20:38:45.764228 6682 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:38:45.764930 6697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:45.765129 6682 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:38:45.765223 6682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "bc42f822d59749eb8ed64ddeab877a47"
format_stamp: "Formatted at 2026-06-19 20:38:45 on dist-test-slave-0t40"
server_key: "3ee307f67c9699a3c0d2c01cb2a38224"
server_key_iv: "7b657a0cfaf8dff5e2b425bb19168ff0"
server_key_version: "encryptionkey@0"
I20260619 20:38:45.765291 6682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:45.772604 6682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:45.774736 6682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:45.774880 6682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:45.778645 6682 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:46257
I20260619 20:38:45.778678 6749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:46257 every 8 connection(s)
I20260619 20:38:45.779021 6682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:38:45.779847 6750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:38:45.781498 6750 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:38:45 dist-test-slave-0t40 krb5kdc[6669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901525, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:38:45.788013 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 6682
I20260619 20:38:45.788130 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:38:45.788369 25850 external_mini_cluster.cc:1428] Setting key 14c92ddc56bcb389eaf8ea369889a80e
I20260619 20:38:45.791472 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:45.782174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35528 (local address 127.25.62.190:46257)
0619 20:38:45.782362 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:45.782371 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:45.782431 (+ 60us) server_negotiation.cc:407] Connection header received
0619 20:38:45.782962 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:45.782964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:45.782987 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:45.783070 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:45.783667 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:45.784471 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:45.785456 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:45.785762 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:45.785861 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:45.787706 (+ 1845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:45.787713 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:45.787713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:45.787719 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:45.789744 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:45.790277 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:45.790279 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:45.790279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:45.790357 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:45.790610 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:45.790610 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:45.790611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:45.790811 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:45.790871 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:45.791119 (+ 248us) server_negotiation.cc:299] Negotiation successful
0619 20:38:45.791308 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":158,"thread_start_us":81,"threads_started":1}
I20260619 20:38:45.791576 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:45.782154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46257 (local address 127.0.0.1:35528)
0619 20:38:45.782332 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:45.782364 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:45.782801 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:45.783126 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:45.783129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:45.783372 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:45.783519 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:45.783523 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:45.784571 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:45.784572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:45.785378 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:45.785381 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:45.785844 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:45.785845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:45.785900 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:45.786322 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:45.786330 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:45.787632 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:45.789850 (+ 2218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:45.789859 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:45.789861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:45.790181 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:45.790427 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:45.790428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:45.790429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:45.790543 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:45.790880 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:45.790886 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:45.790961 (+ 75us) client_negotiation.cc:769] Sending connection context
0619 20:38:45.791061 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 20:38:45.791239 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":95,"threads_started":1}
I20260619 20:38:45.792326 6750 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:38:45.792711 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:45.792564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:40557 (local address 127.0.0.1:49688)
0619 20:38:45.792618 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:45.792680 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 20:38:45.792997 6698 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40557: connect: Connection refused (error 111)
I20260619 20:38:45.793534 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:45.793407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:40557 (local address 127.0.0.1:49700)
0619 20:38:45.793465 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:45.793500 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 20:38:45.793699 6750 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.189:40557: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40557: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.25.62.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.189.keytab.
Entry for principal kudu/127.25.62.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.189.keytab.
I20260619 20:38:45.814249 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.189:40557
--webserver_interface=127.25.62.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.189.keytab
--principal=kudu/127.25.62.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:39463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:46257,127.25.62.189:40557,127.25.62.188:46195
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:38:45.842939 6750 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } attempt: 1
I20260619 20:38:45.843478 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:45.843332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:40557 (local address 127.0.0.1:49714)
0619 20:38:45.843392 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:45.843441 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 20:38:45.843878 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:45.843794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:40557 (local address 127.0.0.1:49726)
0619 20:38:45.843829 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:45.843858 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260619 20:38:45.844015 6750 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.189:40557: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40557: connect: Connection refused (error 111)
W20260619 20:38:45.896600 6757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:45.896812 6757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:45.896834 6757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:45.897926 6757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:38:45.897962 6757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:45.897976 6757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:45.897989 6757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:38:45.898001 6757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:38:45.899474 6757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:39463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:46257,127.25.62.189:40557,127.25.62.188:46195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.189.keytab
--principal=kudu/127.25.62.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.189:40557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.62.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:45.899655 6757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:45.899820 6757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:45.904582 6765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:45.904649 6757 server_base.cc:1034] running on GCE node
W20260619 20:38:45.904577 6763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:45.904573 6762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:45.905079 6757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:45.905400 6757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:45.906576 6757 hybrid_clock.cc:648] HybridClock initialized: now 1781901525906564 us; error 43 us; skew 500 ppm
Jun 19 20:38:45 dist-test-slave-0t40 krb5kdc[6669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901525, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:45.909103 6757 init.cc:377] Logged in from keytab as kudu/127.25.62.189@KRBTEST.COM (short username kudu)
I20260619 20:38:45.909475 6757 webserver.cc:460] Webserver started at http://127.25.62.189:33791/ using document root <none> and password file <none>
I20260619 20:38:45.909663 6757 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:45.909703 6757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:45.909787 6757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:38:45.910760 6757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-1/data/instance:
uuid: "070b77e05e554b1f978846d72768881d"
format_stamp: "Formatted at 2026-06-19 20:38:45 on dist-test-slave-0t40"
server_key: "5a41015461dc6ee137a853c2c5c77f79"
server_key_iv: "ef2c3b09f3c6e00fb75540e0c152b4ac"
server_key_version: "encryptionkey@0"
I20260619 20:38:45.911056 6757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-1/wal/instance:
uuid: "070b77e05e554b1f978846d72768881d"
format_stamp: "Formatted at 2026-06-19 20:38:45 on dist-test-slave-0t40"
server_key: "5a41015461dc6ee137a853c2c5c77f79"
server_key_iv: "ef2c3b09f3c6e00fb75540e0c152b4ac"
server_key_version: "encryptionkey@0"
I20260619 20:38:45.912149 6750 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } attempt: 2
I20260619 20:38:45.912318 6757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:38:45.912581 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:45.912458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:40557 (local address 127.0.0.1:49742)
0619 20:38:45.912505 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:45.912543 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 20:38:45.912922 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:45.912817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:40557 (local address 127.0.0.1:49754)
0619 20:38:45.912866 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:45.912895 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 20:38:45.913013 6750 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.189:40557: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40557: connect: Connection refused (error 111)
I20260619 20:38:45.913189 6772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:45.913364 6757 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:38:45.913450 6757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-1/wal
uuid: "070b77e05e554b1f978846d72768881d"
format_stamp: "Formatted at 2026-06-19 20:38:45 on dist-test-slave-0t40"
server_key: "5a41015461dc6ee137a853c2c5c77f79"
server_key_iv: "ef2c3b09f3c6e00fb75540e0c152b4ac"
server_key_version: "encryptionkey@0"
I20260619 20:38:45.913534 6757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:45.923947 6757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:45.926208 6757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:45.926376 6757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:45.930460 6757 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.189:40557
I20260619 20:38:45.930500 6824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.189:40557 every 8 connection(s)
I20260619 20:38:45.930868 6757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-1/data/info.pb
I20260619 20:38:45.931730 6825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:38:45.933357 6825 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 20:38:45 dist-test-slave-0t40 krb5kdc[6669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901525, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.189@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:38:45.938988 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 6757
I20260619 20:38:45.939065 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-1/wal/instance
I20260619 20:38:45.939270 25850 external_mini_cluster.cc:1428] Setting key 706b2b7e4bf644cb1d8279e8efed5553
I20260619 20:38:45.942181 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:45.934079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35536 (local address 127.25.62.190:46257)
0619 20:38:45.934145 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:45.934148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:45.934334 (+ 186us) server_negotiation.cc:407] Connection header received
0619 20:38:45.934764 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:45.934765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:45.934789 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:45.934852 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:45.935422 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:45.935923 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:45.936839 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:45.937090 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:45.937157 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:45.939228 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:45.939230 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:45.939230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:45.939238 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:45.940702 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:45.941228 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:45.941230 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:45.941230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:45.941293 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:45.941557 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:45.941558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:45.941558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:45.941677 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:45.941751 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:45.942014 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 20:38:45.942093 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 20:38:45.942378 6827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:45.934037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46257 (local address 127.0.0.1:35536)
0619 20:38:45.934190 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:45.934230 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:45.934670 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:45.934920 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:45.934923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:45.935152 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:45.935339 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:45.935342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:45.935998 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:45.935999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:45.936756 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:45.936758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:45.937168 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:45.937169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:45.937272 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:45.937771 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:45.937778 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:45.939125 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:45.940803 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:45.940810 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:45.940812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:45.941140 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:45.941378 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:45.941378 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:45.941379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:45.941488 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:45.941743 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:45.941750 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:45.941888 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 20:38:45.941995 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 20:38:45.942144 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260619 20:38:45.942961 6825 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.25.62.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.188@KRBTEST.COM" created.
Jun 19 20:38:45 dist-test-slave-0t40 krb5kdc[6669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901525, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.189@KRBTEST.COM for kudu/127.25.62.189@KRBTEST.COM
I20260619 20:38:45.953516 6829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:45.943299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49758 (local address 127.25.62.189:40557)
0619 20:38:45.943425 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:45.943429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:45.943486 (+ 57us) server_negotiation.cc:407] Connection header received
0619 20:38:45.943552 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:45.943553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:45.943563 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:45.943621 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:45.943943 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:45.944588 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:45.945597 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:45.945877 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:45.945930 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:45.950468 (+ 4538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:45.950476 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:45.950477 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:45.950484 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:45.952130 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:45.952532 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:45.952535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:45.952535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:45.952594 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:45.952829 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:45.952830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:45.952830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:45.952986 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:45.953129 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:45.953242 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 20:38:45.953373 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":103,"thread_start_us":72,"threads_started":1}
I20260619 20:38:45.953523 6827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:45.943222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:40557 (local address 127.0.0.1:49758)
0619 20:38:45.943425 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:45.943437 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:45.943494 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:45.943646 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:45.943647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:45.943804 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:45.943867 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:45.943869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:45.944675 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:45.944676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:45.945519 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:45.945521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:45.945956 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:45.945957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:45.946041 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:45.946409 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:45.946415 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:45.950354 (+ 3939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:45.952266 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:45.952266 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:45.952267 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:45.952436 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:45.952679 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:45.952679 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:45.952680 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:45.952739 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:45.953090 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:45.953091 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:45.953130 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 20:38:45.953212 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 20:38:45.953426 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":111}
I20260619 20:38:45.954324 6825 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:38:45.954676 6827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:45.954567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:46195 (local address 127.0.0.1:59210)
0619 20:38:45.954610 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:45.954653 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:46195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260619 20:38:45.954839 6775 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.62.188:46195: connect: Connection refused (error 111)
I20260619 20:38:45.955261 6827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:45.955187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:46195 (local address 127.0.0.1:59216)
0619 20:38:45.955213 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:45.955238 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:46195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260619 20:38:45.955387 6825 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.188:46195: Network error: Client connection negotiation failed: client connection to 127.25.62.188:46195: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.188.keytab.
Entry for principal kudu/127.25.62.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.188.keytab.
I20260619 20:38:45.964618 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.188:46195
--webserver_interface=127.25.62.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.188.keytab
--principal=kudu/127.25.62.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:39463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:46257,127.25.62.189:40557,127.25.62.188:46195
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:38:46.004197 6750 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } attempt: 3
I20260619 20:38:46.004643 6825 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } attempt: 1
I20260619 20:38:46.005028 6827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.004891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:46195 (local address 127.0.0.1:59226)
0619 20:38:46.004956 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:46.004989 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:46195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 20:38:46.006072 6827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.005322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:46195 (local address 127.0.0.1:59228)
0619 20:38:46.006012 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:46.006045 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:46195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":675}
W20260619 20:38:46.006181 6825 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.188:46195: Network error: Client connection negotiation failed: client connection to 127.25.62.188:46195: connect: Connection refused (error 111)
Jun 19 20:38:46 dist-test-slave-0t40 krb5kdc[6669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901525, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for kudu/127.25.62.189@KRBTEST.COM
I20260619 20:38:46.011565 6829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.004711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49766 (local address 127.25.62.189:40557)
0619 20:38:46.004756 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:46.004758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:46.004771 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:38:46.004855 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:46.004856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:46.004871 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:46.004924 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:46.005379 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.005902 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.006711 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.006928 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.007015 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:46.009105 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:46.009107 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:46.009107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:46.009113 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:46.010539 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.010909 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.010909 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.010910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.010956 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.011140 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.011140 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.011141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.011237 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:46.011285 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:46.011447 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 20:38:46.011510 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:38:46.011617 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.004640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:40557 (local address 127.0.0.1:49766)
0619 20:38:46.004711 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:46.004723 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:46.004768 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:46.004960 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:46.004962 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:46.005194 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:46.005280 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:46.005283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.005987 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.005988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:46.006612 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:46.006615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.007015 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.007016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:46.007096 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:46.007557 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:46.007562 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:46.008987 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:46.010623 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:46.010624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:46.010624 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:46.010836 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:46.011020 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:46.011020 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:46.011021 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:46.011084 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:46.011297 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:46.011297 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:46.011355 (+ 58us) client_negotiation.cc:769] Sending connection context
0619 20:38:46.011424 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 20:38:46.011521 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 20:38:46.012037 6750 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:38:46.012396 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.012283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:46195 (local address 127.0.0.1:59230)
0619 20:38:46.012338 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:46.012372 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:46195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 20:38:46.012713 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.012629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:46195 (local address 127.0.0.1:59238)
0619 20:38:46.012672 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:46.012697 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:46195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 20:38:46.012806 6750 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.188:46195: Network error: Client connection negotiation failed: client connection to 127.25.62.188:46195: connect: Connection refused (error 111)
I20260619 20:38:46.043955 6750 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } attempt: 1
I20260619 20:38:46.044464 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.044317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:46195 (local address 127.0.0.1:59246)
0619 20:38:46.044380 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:46.044426 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:46195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 20:38:46.044812 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.044727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:46195 (local address 127.0.0.1:59258)
0619 20:38:46.044759 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:46.044789 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:46195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260619 20:38:46.044915 6750 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.188:46195: Network error: Client connection negotiation failed: client connection to 127.25.62.188:46195: connect: Connection refused (error 111)
W20260619 20:38:46.047407 6832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:46.047585 6832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:46.047608 6832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:46.048735 6832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:38:46.048775 6832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:46.048789 6832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:46.048802 6832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:38:46.048853 6832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:38:46.050371 6832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:39463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:46257,127.25.62.189:40557,127.25.62.188:46195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.188.keytab
--principal=kudu/127.25.62.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.188:46195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.62.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:46.050612 6832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:46.050839 6832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:38:46.055644 6832 server_base.cc:1034] running on GCE node
W20260619 20:38:46.055629 6840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:46.055617 6837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:46.055629 6838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:46.056143 6832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:46.056427 6832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:46.057561 6832 hybrid_clock.cc:648] HybridClock initialized: now 1781901526057549 us; error 27 us; skew 500 ppm
Jun 19 20:38:46 dist-test-slave-0t40 krb5kdc[6669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901526, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:46.059959 6832 init.cc:377] Logged in from keytab as kudu/127.25.62.188@KRBTEST.COM (short username kudu)
I20260619 20:38:46.060436 6832 webserver.cc:460] Webserver started at http://127.25.62.188:44159/ using document root <none> and password file <none>
I20260619 20:38:46.060652 6832 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:46.060715 6832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:46.060845 6832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:38:46.061887 6832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-2/data/instance:
uuid: "97f7a0c3520b42e799c663494ac562b4"
format_stamp: "Formatted at 2026-06-19 20:38:46 on dist-test-slave-0t40"
server_key: "b7fcec863c3adc399d8ef9397440741e"
server_key_iv: "c480fb2ba72f73a927fd33b4e5add386"
server_key_version: "encryptionkey@0"
I20260619 20:38:46.062237 6832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-2/wal/instance:
uuid: "97f7a0c3520b42e799c663494ac562b4"
format_stamp: "Formatted at 2026-06-19 20:38:46 on dist-test-slave-0t40"
server_key: "b7fcec863c3adc399d8ef9397440741e"
server_key_iv: "c480fb2ba72f73a927fd33b4e5add386"
server_key_version: "encryptionkey@0"
I20260619 20:38:46.063488 6832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:38:46.064155 6847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:46.064330 6832 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:38:46.064422 6832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-2/wal
uuid: "97f7a0c3520b42e799c663494ac562b4"
format_stamp: "Formatted at 2026-06-19 20:38:46 on dist-test-slave-0t40"
server_key: "b7fcec863c3adc399d8ef9397440741e"
server_key_iv: "c480fb2ba72f73a927fd33b4e5add386"
server_key_version: "encryptionkey@0"
I20260619 20:38:46.064507 6832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:46.074313 6825 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } attempt: 2
I20260619 20:38:46.074748 6827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.074619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:46195 (local address 127.0.0.1:59266)
0619 20:38:46.074684 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:46.074719 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:46195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 20:38:46.075076 6827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.075000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:46195 (local address 127.0.0.1:59280)
0619 20:38:46.075040 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:46.075062 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:46195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 20:38:46.075184 6825 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.188:46195: Network error: Client connection negotiation failed: client connection to 127.25.62.188:46195: connect: Connection refused (error 111)
I20260619 20:38:46.078419 6832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:46.080667 6832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:46.080826 6832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:46.084538 6832 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.188:46195
I20260619 20:38:46.084592 6899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.188:46195 every 8 connection(s)
I20260619 20:38:46.084905 6832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-2/data/info.pb
I20260619 20:38:46.085760 6900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:38:46.087366 6900 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:38:46.089138 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 6832
I20260619 20:38:46.089237 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781901378226483-25850-0/minicluster-data/master-2/wal/instance
I20260619 20:38:46.089607 25850 external_mini_cluster.cc:1428] Setting key 9dd6c6ac1610f613b7a4d3135e6a5e34
Jun 19 20:38:46 dist-test-slave-0t40 krb5kdc[6669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901526, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.188@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:38:46.100855 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.088752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35542 (local address 127.25.62.190:46257)
0619 20:38:46.089613 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:46.089616 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:46.089627 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:38:46.089975 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:46.089976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:46.089990 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:46.090026 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:46.090954 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.091436 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.093032 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.093465 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.093530 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:46.096558 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:46.096560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:46.096560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:46.096568 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:46.099206 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.099741 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.099742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.099743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.099792 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.100096 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.100097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.100097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.100208 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:46.100257 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:46.100659 (+ 402us) server_negotiation.cc:299] Negotiation successful
0619 20:38:46.100746 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":826}
Jun 19 20:38:46 dist-test-slave-0t40 krb5kdc[6669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.188@KRBTEST.COM
I20260619 20:38:46.101418 6902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.088488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46257 (local address 127.0.0.1:35542)
0619 20:38:46.088801 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:46.088848 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:46.089817 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:46.090207 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:46.090210 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:46.090448 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:46.090626 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:46.090630 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.091530 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.091531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:46.092911 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:46.092915 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.093548 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.093549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:46.093692 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:46.094172 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:46.094181 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:46.096464 (+ 2283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:46.099309 (+ 2845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:46.099315 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:46.099317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:46.099653 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:46.099885 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:46.099886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:46.099886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:46.100025 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:46.100276 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:46.100283 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:46.100481 (+ 198us) client_negotiation.cc:769] Sending connection context
0619 20:38:46.100982 (+ 501us) client_negotiation.cc:240] Negotiation successful
0619 20:38:46.101164 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":69,"threads_started":1}
I20260619 20:38:46.102037 6900 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:38:46.105600 6904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.092182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59282 (local address 127.25.62.188:46195)
0619 20:38:46.092399 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:46.092404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:46.092429 (+ 25us) server_negotiation.cc:407] Connection header received
0619 20:38:46.092451 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:46.092454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:46.092473 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:46.092572 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:46.093356 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.095291 (+ 1935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.096149 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.096885 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.099572 (+ 2687us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:46.101994 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:46.101999 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:46.102000 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:46.102006 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:46.103895 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.104314 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.104317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.104317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.104388 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.104607 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.104607 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.104608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.104755 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:46.105256 (+ 501us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:46.105303 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 20:38:46.105486 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":70,"threads_started":1}
Jun 19 20:38:46 dist-test-slave-0t40 krb5kdc[6669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901526, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.188@KRBTEST.COM for kudu/127.25.62.189@KRBTEST.COM
I20260619 20:38:46.111426 6902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.102625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:40557 (local address 127.0.0.1:49778)
0619 20:38:46.102670 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:46.102679 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:46.102714 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:46.103091 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:46.103093 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:46.103377 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:46.103415 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:46.103417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.104332 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.104333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:46.105191 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:46.105194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.106099 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.106100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:46.106167 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:46.106592 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:46.106598 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:46.108116 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:46.110239 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:46.110240 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:46.110240 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:46.110448 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:46.110699 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:46.110700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:46.110700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:46.110773 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:46.111112 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:46.111113 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:46.111161 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 20:38:46.111236 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 20:38:46.111326 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 20:38:46.111483 6829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.102710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49778 (local address 127.25.62.189:40557)
0619 20:38:46.102755 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:46.102757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:46.102765 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:38:46.102790 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:46.102791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:46.102801 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:46.102856 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:46.103574 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.104251 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.105297 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.105758 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.105811 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:46.108277 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:46.108279 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:46.108279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:46.108287 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:46.110145 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.110530 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.110532 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.110532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.110614 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.110885 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.110886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.110887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.111031 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:46.111105 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:46.111281 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 20:38:46.111377 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Jun 19 20:38:46 dist-test-slave-0t40 krb5kdc[6669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:38:46.112365 6900 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:38:46.115638 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.106599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35550 (local address 127.25.62.190:46257)
0619 20:38:46.106635 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:46.106637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:46.106644 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:38:46.106676 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:46.106677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:46.106685 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:46.106739 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:46.107306 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.108007 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.109268 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.109568 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.109647 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:46.112216 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:46.112218 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:46.112218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:46.112226 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:46.114164 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.114589 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.114590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.114591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.114649 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.114856 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.114857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.114858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.114976 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:46.115153 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:46.115399 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 20:38:46.115483 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Jun 19 20:38:46 dist-test-slave-0t40 krb5kdc[6669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901526, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.188@KRBTEST.COM for kudu/127.25.62.188@KRBTEST.COM
I20260619 20:38:46.120101 6750 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } attempt: 2
I20260619 20:38:46.121232 6902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.112625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:46195 (local address 127.0.0.1:59286)
0619 20:38:46.112688 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:46.112697 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:46.112731 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:46.113239 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:46.113239 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:46.113388 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:46.113431 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:46.113433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.114150 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.114151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:46.114986 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:46.114989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.115472 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.115473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:46.115522 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:46.115945 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:46.115949 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:46.117790 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:46.119694 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:46.119694 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:46.119694 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:46.119921 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:46.120501 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:46.120502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:46.120502 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:46.120573 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:46.120881 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:46.120882 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:46.120922 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 20:38:46.120989 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 20:38:46.121135 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 20:38:46.121703 6904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.112794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59286 (local address 127.25.62.188:46195)
0619 20:38:46.113088 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:46.113090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:46.113098 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:38:46.113115 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:46.113116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:46.113147 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:46.113204 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:46.113518 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.114074 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.115076 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.115393 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.115464 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:46.117880 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:46.117881 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:46.117882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:46.117888 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:46.119619 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.119992 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.119993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.119993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.120093 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.120646 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.120646 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.120647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.120785 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:46.120845 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:46.121540 (+ 695us) server_negotiation.cc:299] Negotiation successful
0619 20:38:46.121612 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277}
Jun 19 20:38:46 dist-test-slave-0t40 krb5kdc[6669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.189@KRBTEST.COM
I20260619 20:38:46.124007 6900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4: Bootstrap starting.
I20260619 20:38:46.125126 6900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4: Neither blocks nor log segments found. Creating new log.
I20260619 20:38:46.125469 6900 log.cc:824] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4: Log is configured to *not* fsync() on all Append() calls
Jun 19 20:38:46 dist-test-slave-0t40 krb5kdc[6669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901525, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for kudu/127.25.62.188@KRBTEST.COM
I20260619 20:38:46.126376 6900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4: No bootstrap required, opened a new log
I20260619 20:38:46.128789 6900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } } peers { permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } } peers { permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } }
I20260619 20:38:46.129102 6900 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:38:46.129211 6900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97f7a0c3520b42e799c663494ac562b4, State: Initialized, Role: FOLLOWER
I20260619 20:38:46.129431 6829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.116573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49784 (local address 127.25.62.189:40557)
0619 20:38:46.116622 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:46.116624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:46.116697 (+ 73us) server_negotiation.cc:407] Connection header received
0619 20:38:46.116801 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:46.116802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:46.116811 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:46.116858 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:46.117541 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.118249 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.119184 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.119526 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.119604 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:46.125766 (+ 6162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:46.125768 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:46.125768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:46.125776 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:46.128012 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.128420 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.128421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.128421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.128478 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.128683 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.128684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.128684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.128806 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:46.128968 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:46.129241 (+ 273us) server_negotiation.cc:299] Negotiation successful
0619 20:38:46.129326 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:38:46.129446 6900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } } peers { permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } } peers { permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } }
I20260619 20:38:46.129848 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.120345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:46195 (local address 127.0.0.1:59298)
0619 20:38:46.120400 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:46.120410 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:46.120446 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:46.120775 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:46.120776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:46.121061 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:46.121138 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:46.121141 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.122627 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.122628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:46.123782 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:46.123785 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.124384 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.124386 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:46.124449 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:46.124871 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:46.124876 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:46.126505 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:46.128515 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:46.128516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:46.128517 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:46.128752 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:46.129028 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:46.129029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:46.129030 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:46.129128 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:46.129501 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:46.129502 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:46.129551 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 20:38:46.129606 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 20:38:46.129708 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 20:38:46.130206 6907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } } peers { permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } } peers { permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } } }
I20260619 20:38:46.130275 6900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:38:46.130312 6907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:38:46.130280 6905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.120446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59298 (local address 127.25.62.188:46195)
0619 20:38:46.120619 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:46.120621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:46.120629 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:38:46.120647 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:46.120648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:46.120657 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:46.120725 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:46.121299 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.122495 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.123896 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.124281 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.124600 (+ 319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:46.126605 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:46.126606 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:46.126606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:46.126613 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:46.128412 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.128884 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.128886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.128886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.128946 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.129212 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.129213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.129213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.129414 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:46.130016 (+ 602us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:46.130060 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 20:38:46.130143 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":146,"threads_started":1}
I20260619 20:38:46.130868 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260619 20:38:46.132459 6918 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:38:46.132573 6918 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:38:46.133529 6750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47: Bootstrap starting.
I20260619 20:38:46.134231 6750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47: Neither blocks nor log segments found. Creating new log.
I20260619 20:38:46.134554 6750 log.cc:824] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47: Log is configured to *not* fsync() on all Append() calls
I20260619 20:38:46.138893 6750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47: No bootstrap required, opened a new log
I20260619 20:38:46.135298 6832 master_runner.cc:186] Error getting master registration for 127.25.62.190:46257: OK, instance_id { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" instance_seqno: 1781901525777601 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 20:38:46.139437 6832 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 20:38:46.140165 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.132373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35558 (local address 127.25.62.190:46257)
0619 20:38:46.132409 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:46.132412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:46.132820 (+ 408us) server_negotiation.cc:407] Connection header received
0619 20:38:46.132882 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:46.132883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:46.132893 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:46.132949 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:46.133706 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.134405 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.135408 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.135678 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.135722 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:46.137324 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:46.137326 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:46.137326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:46.137333 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:46.138922 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.139298 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.139300 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.139300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.139355 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.139573 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.139574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.139575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.139703 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:46.139785 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:46.140008 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 20:38:46.140080 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:38:46.141352 6750 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } } peers { permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } } peers { permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } }
I20260619 20:38:46.141492 6750 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:38:46.141535 6750 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc42f822d59749eb8ed64ddeab877a47, State: Initialized, Role: FOLLOWER
I20260619 20:38:46.141656 6750 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } } peers { permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } } peers { permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } }
I20260619 20:38:46.141979 6750 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:38:46.144088 6928 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } } peers { permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } } peers { permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } } }
I20260619 20:38:46.144357 6928 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:38:46.146021 6905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.134858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59306 (local address 127.25.62.188:46195)
0619 20:38:46.134950 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:46.134952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:46.134958 (+ 6us) server_negotiation.cc:407] Connection header received
0619 20:38:46.134975 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:46.134976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:46.134984 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:46.135030 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:46.135797 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.136611 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.137741 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.138122 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.138166 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:46.142758 (+ 4592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:46.142759 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:46.142760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:46.142766 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:46.144447 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.144874 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.144875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.144875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.144928 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.145249 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.145250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.145250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.145398 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:46.145508 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:46.145860 (+ 352us) server_negotiation.cc:299] Negotiation successful
0619 20:38:46.145937 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260619 20:38:46.147991 6939 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:38:46.148044 6939 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:38:46.145838 6682 master_runner.cc:186] Error getting master registration for 127.25.62.189:40557: OK, instance_id { permanent_uuid: "070b77e05e554b1f978846d72768881d" instance_seqno: 1781901525929340 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 20:38:46.148609 6682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 20:38:46.148624 6829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.132389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49788 (local address 127.25.62.189:40557)
0619 20:38:46.139512 (+ 7123us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:46.139514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:46.139521 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:38:46.139542 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:46.139543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:46.139554 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:46.139606 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:46.141852 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.142544 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.143585 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.143885 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.143960 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:46.145357 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:46.145358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:46.145358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:46.145365 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:46.146979 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.147485 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.147486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.147486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.147545 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.147759 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.147760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.147760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.147879 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:46.147983 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:46.148465 (+ 482us) server_negotiation.cc:299] Negotiation successful
0619 20:38:46.148538 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7098}
I20260619 20:38:46.166365 6825 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } attempt: 3
Jun 19 20:38:46 dist-test-slave-0t40 krb5kdc[6669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901525, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.189@KRBTEST.COM for kudu/127.25.62.188@KRBTEST.COM
I20260619 20:38:46.173177 6905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.166698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59312 (local address 127.25.62.188:46195)
0619 20:38:46.166732 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:46.166734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:46.166742 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:38:46.166808 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:46.166808 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:46.166817 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:46.166899 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:46.167156 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.167770 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.168425 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.168661 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.168745 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:46.170552 (+ 1807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:46.170553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:46.170553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:46.170560 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:46.171955 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.172365 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.172366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.172366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.172421 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:46.172635 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:46.172635 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:46.172636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:46.172757 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:46.172839 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:46.172995 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 20:38:46.173093 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:38:46.173177 6827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:46.166629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:46195 (local address 127.0.0.1:59312)
0619 20:38:46.166678 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:46.166689 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:46.166730 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:46.166903 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:46.166903 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:46.167044 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:46.167080 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:46.167081 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.167841 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.167842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:46.168331 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:46.168333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:46.168746 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:46.168747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:46.168827 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:46.169318 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:46.169334 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:46.170466 (+ 1132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:46.172054 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:46.172055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:46.172055 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:46.172269 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:46.172496 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:46.172497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:46.172497 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:46.172560 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:46.172840 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:46.172841 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:46.172883 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 20:38:46.172971 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 20:38:46.173085 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 20:38:46.174672 6825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d: Bootstrap starting.
I20260619 20:38:46.175279 6825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d: Neither blocks nor log segments found. Creating new log.
I20260619 20:38:46.175571 6825 log.cc:824] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d: Log is configured to *not* fsync() on all Append() calls
I20260619 20:38:46.176296 6825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d: No bootstrap required, opened a new log
I20260619 20:38:46.177862 6825 raft_consensus.cc:348] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } } peers { permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } } peers { permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } }
I20260619 20:38:46.177982 6825 raft_consensus.cc:374] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:38:46.178023 6825 raft_consensus.cc:729] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 070b77e05e554b1f978846d72768881d, State: Initialized, Role: FOLLOWER
I20260619 20:38:46.178146 6825 consensus_queue.cc:260] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } } peers { permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } } peers { permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } }
I20260619 20:38:46.178397 6942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } } peers { permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } } peers { permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } } }
I20260619 20:38:46.178449 6825 sys_catalog.cc:564] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:38:46.178521 6942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d [sys.catalog]: This master's current role is: FOLLOWER
W20260619 20:38:46.180519 6953 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:38:46.180621 6953 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:38:46.182531 6757 master_runner.cc:263] No leader master found from master 070b77e05e554b1f978846d72768881d
I20260619 20:38:46.182581 6757 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260619 20:38:46.275622 6703 tablet.cc:2369] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:38:46.318367 6928 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:38:46.318521 6928 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } } peers { permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } } peers { permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } }
I20260619 20:38:46.318809 6928 leader_election.cc:288] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 070b77e05e554b1f978846d72768881d (127.25.62.189:40557), 97f7a0c3520b42e799c663494ac562b4 (127.25.62.188:46195)
I20260619 20:38:46.319128 6799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc42f822d59749eb8ed64ddeab877a47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "070b77e05e554b1f978846d72768881d" is_pre_election: true
I20260619 20:38:46.319114 6874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc42f822d59749eb8ed64ddeab877a47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97f7a0c3520b42e799c663494ac562b4" is_pre_election: true
I20260619 20:38:46.319272 6874 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc42f822d59749eb8ed64ddeab877a47 in term 0.
I20260619 20:38:46.319279 6799 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc42f822d59749eb8ed64ddeab877a47 in term 0.
I20260619 20:38:46.319478 6700 leader_election.cc:302] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97f7a0c3520b42e799c663494ac562b4, bc42f822d59749eb8ed64ddeab877a47; no voters:
I20260619 20:38:46.319619 6928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:38:46.319665 6928 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:38:46.319695 6928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:38:46.320428 6928 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } } peers { permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } } peers { permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } }
I20260619 20:38:46.320564 6928 leader_election.cc:288] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [CANDIDATE]: Term 1 election: Requested vote from peers 070b77e05e554b1f978846d72768881d (127.25.62.189:40557), 97f7a0c3520b42e799c663494ac562b4 (127.25.62.188:46195)
I20260619 20:38:46.320745 6874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc42f822d59749eb8ed64ddeab877a47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97f7a0c3520b42e799c663494ac562b4"
I20260619 20:38:46.320786 6799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc42f822d59749eb8ed64ddeab877a47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "070b77e05e554b1f978846d72768881d"
I20260619 20:38:46.320832 6874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:38:46.320848 6799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:38:46.321620 6874 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc42f822d59749eb8ed64ddeab877a47 in term 1.
I20260619 20:38:46.321751 6799 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc42f822d59749eb8ed64ddeab877a47 in term 1.
I20260619 20:38:46.321923 6700 leader_election.cc:302] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97f7a0c3520b42e799c663494ac562b4, bc42f822d59749eb8ed64ddeab877a47; no voters:
I20260619 20:38:46.322028 6928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:38:46.322170 6928 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [term 1 LEADER]: Becoming Leader. State: Replica: bc42f822d59749eb8ed64ddeab877a47, State: Running, Role: LEADER
I20260619 20:38:46.322273 6928 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } } peers { permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } } peers { permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } }
I20260619 20:38:46.322695 6955 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc42f822d59749eb8ed64ddeab877a47. Latest consensus state: current_term: 1 leader_uuid: "bc42f822d59749eb8ed64ddeab877a47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } } peers { permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } } peers { permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } } }
I20260619 20:38:46.322837 6955 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [sys.catalog]: This master's current role is: LEADER
I20260619 20:38:46.323067 6957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:38:46.323262 6957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:38:46.324515 6874 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4 [term 1 FOLLOWER]: Refusing update from remote peer bc42f822d59749eb8ed64ddeab877a47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:38:46.324514 6799 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d [term 1 FOLLOWER]: Refusing update from remote peer bc42f822d59749eb8ed64ddeab877a47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:38:46.324851 6928 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:38:46.324992 6955 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:38:46.326107 6907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc42f822d59749eb8ed64ddeab877a47. Latest consensus state: current_term: 1 leader_uuid: "bc42f822d59749eb8ed64ddeab877a47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } } peers { permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } } peers { permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } } }
I20260619 20:38:46.326107 6942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc42f822d59749eb8ed64ddeab877a47. Latest consensus state: current_term: 1 leader_uuid: "bc42f822d59749eb8ed64ddeab877a47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } } peers { permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } } peers { permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } } }
I20260619 20:38:46.326188 6907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:38:46.326188 6942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:38:46.327416 6928 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc42f822d59749eb8ed64ddeab877a47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } } peers { permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } } peers { permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } } }
I20260619 20:38:46.327495 6928 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [sys.catalog]: This master's current role is: LEADER
I20260619 20:38:46.327637 6942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc42f822d59749eb8ed64ddeab877a47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } } peers { permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } } peers { permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } } }
I20260619 20:38:46.327713 6942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:38:46.327822 6960 mvcc.cc:205] Tried to move back new op lower bound from 7298668651823370240 to 7298668651816603648. Current Snapshot: MvccSnapshot[applied={T|T < 7298668651823370240}]
I20260619 20:38:46.328459 6955 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc42f822d59749eb8ed64ddeab877a47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } } peers { permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } } peers { permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } } }
I20260619 20:38:46.328548 6955 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47 [sys.catalog]: This master's current role is: LEADER
I20260619 20:38:46.328944 6907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc42f822d59749eb8ed64ddeab877a47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc42f822d59749eb8ed64ddeab877a47" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46257 } } peers { permanent_uuid: "070b77e05e554b1f978846d72768881d" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40557 } } peers { permanent_uuid: "97f7a0c3520b42e799c663494ac562b4" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 46195 } } }
I20260619 20:38:46.329031 6907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:38:46.329639 6957 catalog_manager.cc:1347] Generated new cluster ID: a40418d5fd4c43d9853d30e66747766c
I20260619 20:38:46.329697 6957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:38:46.343979 6957 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:38:46.344657 6957 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:38:46.347769 6957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc42f822d59749eb8ed64ddeab877a47: Generated new TSK 0
I20260619 20:38:46.347975 6957 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:38:47.133100 6918 catalog_manager.cc:1259] Loaded cluster ID: a40418d5fd4c43d9853d30e66747766c
I20260619 20:38:47.133173 6918 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4: loading cluster ID for follower catalog manager: success
I20260619 20:38:47.133996 6918 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4: acquiring CA information for follower catalog manager: success
I20260619 20:38:47.134295 6918 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 97f7a0c3520b42e799c663494ac562b4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:38:47.181088 6953 catalog_manager.cc:1259] Loaded cluster ID: a40418d5fd4c43d9853d30e66747766c
I20260619 20:38:47.181166 6953 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d: loading cluster ID for follower catalog manager: success
I20260619 20:38:47.182030 6953 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d: acquiring CA information for follower catalog manager: success
I20260619 20:38:47.182353 6953 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 070b77e05e554b1f978846d72768881d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:38:47.400698 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 6682
I20260619 20:38:47.405742 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 6757
I20260619 20:38:47.411032 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 6832
2026-06-19T20:38:47Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2905 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:38:47 dist-test-slave-0t40 krb5kdc[6972](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:38:47 dist-test-slave-0t40 krb5kdc[6972](info): set up 2 sockets
Jun 19 20:38:47 dist-test-slave-0t40 krb5kdc[6972](info): commencing operation
krb5kdc: starting...
W20260619 20:38:48.445511 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:38:48 dist-test-slave-0t40 krb5kdc[6972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:38:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:38:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:38:48.561993 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:39807
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:39377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:39807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:38:48.640890 6985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:48.641192 6985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:48.641234 6985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:48.642379 6985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:38:48.642448 6985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:48.642472 6985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:48.642493 6985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:38:48.642520 6985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:38:48.643946 6985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:39377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:39807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:39807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:48.644212 6985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:48.644434 6985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:48.649446 6993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:48.649523 6991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:48.649451 6990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:48.649757 6985 server_base.cc:1034] running on GCE node
I20260619 20:38:48.649987 6985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:48.650231 6985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:48.651394 6985 hybrid_clock.cc:648] HybridClock initialized: now 1781901528651373 us; error 38 us; skew 500 ppm
Jun 19 20:38:48 dist-test-slave-0t40 krb5kdc[6972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901528, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:48.653872 6985 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:38:48.654356 6985 webserver.cc:460] Webserver started at http://127.25.62.190:33641/ using document root <none> and password file <none>
I20260619 20:38:48.654587 6985 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:48.654655 6985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:48.654808 6985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:38:48.655865 6985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "d902fe96cb7f4d7a9f48b925c6dd3a63"
format_stamp: "Formatted at 2026-06-19 20:38:48 on dist-test-slave-0t40"
server_key: "98c6e07d8a03f2732c1ea945470ca062"
server_key_iv: "d556f5900d4f9df560534d6b83cfd0f1"
server_key_version: "encryptionkey@0"
I20260619 20:38:48.656236 6985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "d902fe96cb7f4d7a9f48b925c6dd3a63"
format_stamp: "Formatted at 2026-06-19 20:38:48 on dist-test-slave-0t40"
server_key: "98c6e07d8a03f2732c1ea945470ca062"
server_key_iv: "d556f5900d4f9df560534d6b83cfd0f1"
server_key_version: "encryptionkey@0"
I20260619 20:38:48.657629 6985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 20:38:48.658289 7000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:48.658463 6985 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:38:48.658558 6985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "d902fe96cb7f4d7a9f48b925c6dd3a63"
format_stamp: "Formatted at 2026-06-19 20:38:48 on dist-test-slave-0t40"
server_key: "98c6e07d8a03f2732c1ea945470ca062"
server_key_iv: "d556f5900d4f9df560534d6b83cfd0f1"
server_key_version: "encryptionkey@0"
I20260619 20:38:48.658638 6985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:48.671512 6985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:48.673441 6985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:48.673594 6985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:48.677712 6985 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:39807
I20260619 20:38:48.677758 7052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:39807 every 8 connection(s)
I20260619 20:38:48.678117 6985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:38:48.678928 7053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:38:48.681180 7053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63: Bootstrap starting.
I20260619 20:38:48.681731 7053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63: Neither blocks nor log segments found. Creating new log.
I20260619 20:38:48.682022 7053 log.cc:824] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63: Log is configured to *not* fsync() on all Append() calls
I20260619 20:38:48.682690 7053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63: No bootstrap required, opened a new log
I20260619 20:38:48.684229 7053 raft_consensus.cc:348] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d902fe96cb7f4d7a9f48b925c6dd3a63" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 39807 } }
I20260619 20:38:48.684365 7053 raft_consensus.cc:374] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:38:48.684388 7053 raft_consensus.cc:729] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d902fe96cb7f4d7a9f48b925c6dd3a63, State: Initialized, Role: FOLLOWER
I20260619 20:38:48.684543 7053 consensus_queue.cc:260] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d902fe96cb7f4d7a9f48b925c6dd3a63" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 39807 } }
I20260619 20:38:48.684603 7053 raft_consensus.cc:388] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:38:48.684625 7053 raft_consensus.cc:482] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:38:48.684675 7053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:38:48.685256 7053 raft_consensus.cc:504] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d902fe96cb7f4d7a9f48b925c6dd3a63" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 39807 } }
I20260619 20:38:48.685357 7053 leader_election.cc:302] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d902fe96cb7f4d7a9f48b925c6dd3a63; no voters:
I20260619 20:38:48.685523 7053 leader_election.cc:288] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:38:48.685570 7056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:38:48.685802 7053 sys_catalog.cc:564] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:38:48.685801 7056 raft_consensus.cc:686] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63 [term 1 LEADER]: Becoming Leader. State: Replica: d902fe96cb7f4d7a9f48b925c6dd3a63, State: Running, Role: LEADER
I20260619 20:38:48.685931 7056 consensus_queue.cc:237] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d902fe96cb7f4d7a9f48b925c6dd3a63" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 39807 } }
I20260619 20:38:48.686352 7058 sys_catalog.cc:455] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d902fe96cb7f4d7a9f48b925c6dd3a63. Latest consensus state: current_term: 1 leader_uuid: "d902fe96cb7f4d7a9f48b925c6dd3a63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d902fe96cb7f4d7a9f48b925c6dd3a63" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 39807 } } }
I20260619 20:38:48.686434 7058 sys_catalog.cc:458] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63 [sys.catalog]: This master's current role is: LEADER
I20260619 20:38:48.686426 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 6985
I20260619 20:38:48.686525 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:38:48.686479 7057 sys_catalog.cc:455] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d902fe96cb7f4d7a9f48b925c6dd3a63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d902fe96cb7f4d7a9f48b925c6dd3a63" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 39807 } } }
I20260619 20:38:48.686554 7057 sys_catalog.cc:458] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63 [sys.catalog]: This master's current role is: LEADER
I20260619 20:38:48.686712 25850 external_mini_cluster.cc:1428] Setting key b2ecca57a029d8590634836f6d268a48
I20260619 20:38:48.686859 7060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:38:48.687361 7060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:38:48.689553 7060 catalog_manager.cc:1347] Generated new cluster ID: dba95e21e14a4f48a2fa1e3df0cacfef
I20260619 20:38:48.689611 7060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:38:48 dist-test-slave-0t40 krb5kdc[6972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:38:48.699452 7069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:48.687351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46324 (local address 127.25.62.190:39807)
0619 20:38:48.687935 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:48.687943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:48.687975 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:38:48.688168 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:48.688176 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:48.688244 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:48.688420 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:48.689127 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:48.690459 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:48.691491 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:48.692100 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:48.692214 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:48.695093 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:48.695097 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:48.695098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:48.695109 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:48.697754 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:48.698186 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:48.698188 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:48.698189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:48.698309 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:48.698530 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:48.698531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:48.698531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:48.698824 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:48.698972 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:48.699094 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 20:38:48.699222 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":90,"threads_started":1}
I20260619 20:38:48.700397 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:38:48.707928 7060 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:38:48.708412 7060 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:38:48.713193 7060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d902fe96cb7f4d7a9f48b925c6dd3a63: Generated new TSK 0
I20260619 20:38:48.713392 7060 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:38:48.754386 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 6985
2026-06-19T20:38:48Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1340 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:38:48 dist-test-slave-0t40 krb5kdc[7080](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:38:48 dist-test-slave-0t40 krb5kdc[7080](info): set up 2 sockets
Jun 19 20:38:48 dist-test-slave-0t40 krb5kdc[7080](info): commencing operation
krb5kdc: starting...
W20260619 20:38:49.785454 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:38:49 dist-test-slave-0t40 krb5kdc[7080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:38:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:38:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:38:49.904006 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:46449
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:46449
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:38:49.983884 7093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:49.984089 7093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:49.984112 7093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:49.985277 7093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:38:49.985319 7093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:49.985333 7093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:49.985347 7093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:38:49.985358 7093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:38:49.986817 7093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:46449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:46449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:49.987072 7093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:49.987295 7093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:49.992077 7101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:49.992072 7099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:49.992228 7098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:49.992365 7093 server_base.cc:1034] running on GCE node
I20260619 20:38:49.992563 7093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:49.992841 7093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:49.993974 7093 hybrid_clock.cc:648] HybridClock initialized: now 1781901529993957 us; error 38 us; skew 500 ppm
Jun 19 20:38:49 dist-test-slave-0t40 krb5kdc[7080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901529, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:49.996335 7093 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:38:49.996770 7093 webserver.cc:460] Webserver started at http://127.25.62.190:41215/ using document root <none> and password file <none>
I20260619 20:38:49.996959 7093 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:49.996999 7093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:49.997139 7093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:38:49.998183 7093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "0bd0501973194ca1b6934e64e4e18477"
format_stamp: "Formatted at 2026-06-19 20:38:49 on dist-test-slave-0t40"
server_key: "66802977e17c225f20b886a8c457dabd"
server_key_iv: "d0320f6cd9752cb4f15e1ed4f1f0108d"
server_key_version: "encryptionkey@0"
I20260619 20:38:49.998484 7093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "0bd0501973194ca1b6934e64e4e18477"
format_stamp: "Formatted at 2026-06-19 20:38:49 on dist-test-slave-0t40"
server_key: "66802977e17c225f20b886a8c457dabd"
server_key_iv: "d0320f6cd9752cb4f15e1ed4f1f0108d"
server_key_version: "encryptionkey@0"
I20260619 20:38:49.999727 7093 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:38:50.000447 7108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:50.000658 7093 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:38:50.000761 7093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "0bd0501973194ca1b6934e64e4e18477"
format_stamp: "Formatted at 2026-06-19 20:38:49 on dist-test-slave-0t40"
server_key: "66802977e17c225f20b886a8c457dabd"
server_key_iv: "d0320f6cd9752cb4f15e1ed4f1f0108d"
server_key_version: "encryptionkey@0"
I20260619 20:38:50.000845 7093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:50.007726 7093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:50.009511 7093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:50.009661 7093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:50.013733 7093 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:46449
I20260619 20:38:50.013779 7160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:46449 every 8 connection(s)
I20260619 20:38:50.014129 7093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:38:50.015041 7161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:38:50.017235 7161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477: Bootstrap starting.
I20260619 20:38:50.017797 7161 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477: Neither blocks nor log segments found. Creating new log.
I20260619 20:38:50.018097 7161 log.cc:824] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477: Log is configured to *not* fsync() on all Append() calls
I20260619 20:38:50.018579 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 7093
I20260619 20:38:50.018687 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:38:50.018867 25850 external_mini_cluster.cc:1428] Setting key 4caa035dcb5608750a92ac82ee7df097
I20260619 20:38:50.018963 7161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477: No bootstrap required, opened a new log
I20260619 20:38:50.020764 7161 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd0501973194ca1b6934e64e4e18477" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46449 } }
I20260619 20:38:50.020924 7161 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:38:50.020968 7161 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0501973194ca1b6934e64e4e18477, State: Initialized, Role: FOLLOWER
I20260619 20:38:50.021124 7161 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd0501973194ca1b6934e64e4e18477" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46449 } }
I20260619 20:38:50.021224 7161 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:38:50.021262 7161 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:38:50.021312 7161 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:38:50.022138 7161 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd0501973194ca1b6934e64e4e18477" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46449 } }
I20260619 20:38:50.022253 7161 leader_election.cc:302] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bd0501973194ca1b6934e64e4e18477; no voters:
I20260619 20:38:50.022409 7161 leader_election.cc:288] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:38:50.022445 7166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:38:50.022554 7166 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 1 LEADER]: Becoming Leader. State: Replica: 0bd0501973194ca1b6934e64e4e18477, State: Running, Role: LEADER
I20260619 20:38:50.022631 7166 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd0501973194ca1b6934e64e4e18477" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46449 } }
I20260619 20:38:50.022663 7161 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:38:50.022927 7166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bd0501973194ca1b6934e64e4e18477. Latest consensus state: current_term: 1 leader_uuid: "0bd0501973194ca1b6934e64e4e18477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd0501973194ca1b6934e64e4e18477" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46449 } } }
I20260619 20:38:50.023056 7166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [sys.catalog]: This master's current role is: LEADER
I20260619 20:38:50.022943 7167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bd0501973194ca1b6934e64e4e18477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd0501973194ca1b6934e64e4e18477" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46449 } } }
I20260619 20:38:50.023222 7167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [sys.catalog]: This master's current role is: LEADER
I20260619 20:38:50.023371 7171 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:38:50.023865 7171 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:38:50.025830 7171 catalog_manager.cc:1347] Generated new cluster ID: 571a2301f0604d31afe00008dc4b5c88
I20260619 20:38:50.025885 7171 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:38:50 dist-test-slave-0t40 krb5kdc[7080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:38:50.031246 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.019374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60664 (local address 127.25.62.190:46449)
0619 20:38:50.019549 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:50.019557 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:50.019581 (+ 24us) server_negotiation.cc:407] Connection header received
0619 20:38:50.020755 (+ 1174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:50.020762 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:50.020813 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:50.020984 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:50.021688 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.022793 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.023914 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.024601 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.025003 (+ 402us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:50.027442 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:50.027446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:50.027446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:50.027457 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:50.029400 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:50.029751 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:50.029754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:50.029754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:50.029880 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:50.030055 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:50.030056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:50.030056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:50.030323 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:50.030837 (+ 514us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:50.030902 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 20:38:50.031030 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":96,"threads_started":1}
I20260619 20:38:50.041473 7171 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:38:50.041970 7171 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:38:50.045048 7171 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477: Generated new TSK 0
I20260619 20:38:50.045222 7171 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
I20260619 20:38:50.072106 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:46449
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:38:50.152577 7187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:50.152791 7187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:50.152814 7187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:50.154035 7187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:50.154079 7187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:50.154105 7187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:38:50.155565 7187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:46449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:50.155805 7187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:50.156057 7187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:50.160974 7195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:50.161011 7193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:50.161072 7192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:50.161283 7187 server_base.cc:1034] running on GCE node
I20260619 20:38:50.161481 7187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:50.161674 7187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:50.162829 7187 hybrid_clock.cc:648] HybridClock initialized: now 1781901530162811 us; error 34 us; skew 500 ppm
Jun 19 20:38:50 dist-test-slave-0t40 krb5kdc[7080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901530, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:50.165681 7187 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:38:50.166126 7187 webserver.cc:460] Webserver started at http://127.25.62.129:43001/ using document root <none> and password file <none>
I20260619 20:38:50.166363 7187 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:50.166407 7187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:50.166570 7187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:38:50.167647 7187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "611432b752984ba58352a16639bc031b"
format_stamp: "Formatted at 2026-06-19 20:38:50 on dist-test-slave-0t40"
server_key: "2e0797f7a91a232aed591f9d8cf2398e"
server_key_iv: "aa1f652056722269aa80a9a560849191"
server_key_version: "encryptionkey@0"
I20260619 20:38:50.168001 7187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "611432b752984ba58352a16639bc031b"
format_stamp: "Formatted at 2026-06-19 20:38:50 on dist-test-slave-0t40"
server_key: "2e0797f7a91a232aed591f9d8cf2398e"
server_key_iv: "aa1f652056722269aa80a9a560849191"
server_key_version: "encryptionkey@0"
I20260619 20:38:50.169343 7187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:38:50.170056 7202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:50.170288 7187 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:38:50.170392 7187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "611432b752984ba58352a16639bc031b"
format_stamp: "Formatted at 2026-06-19 20:38:50 on dist-test-slave-0t40"
server_key: "2e0797f7a91a232aed591f9d8cf2398e"
server_key_iv: "aa1f652056722269aa80a9a560849191"
server_key_version: "encryptionkey@0"
I20260619 20:38:50.170477 7187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:50.182044 7187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:50.184266 7187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:50.184433 7187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:50.184653 7187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:38:50.184955 7187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:38:50.185009 7187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:50.185041 7187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:38:50.185133 7187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:50.190650 7187 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:43075
I20260619 20:38:50.190724 7315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:43075 every 8 connection(s)
I20260619 20:38:50.191032 7187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
Jun 19 20:38:50 dist-test-slave-0t40 krb5kdc[7080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901530, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:38:50.197412 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 7187
I20260619 20:38:50.197530 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:38:50.197734 25850 external_mini_cluster.cc:1428] Setting key 042dbddd83300900c77335b7a6d813a4
I20260619 20:38:50.200431 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.191777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:59675 (local address 127.25.62.190:46449)
0619 20:38:50.191826 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:50.191829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:50.192059 (+ 230us) server_negotiation.cc:407] Connection header received
0619 20:38:50.192496 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:50.192497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:50.192511 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:50.192573 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:50.193497 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.193990 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.194932 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.195127 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.195181 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:50.197233 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:50.197234 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:50.197235 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:50.197246 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:50.198983 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:50.199400 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:50.199401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:50.199402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:50.199462 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:50.199744 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:50.199745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:50.199745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:50.199894 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:50.199966 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:50.200293 (+ 327us) server_negotiation.cc:299] Negotiation successful
0619 20:38:50.200359 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:38:50.200726 7318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.191762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:59675)
0619 20:38:50.191928 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:50.191963 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:50.192409 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:50.192624 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:50.192627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:50.193190 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:50.193381 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:50.193414 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.194072 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.194072 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:50.194858 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:50.194859 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.195200 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.195201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:50.195295 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:50.195734 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:50.195741 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:50.197138 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:50.199076 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:50.199083 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:50.199085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:50.199302 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:50.199532 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:50.199532 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:50.199533 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:50.199653 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:50.199972 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:50.199979 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:50.200149 (+ 170us) client_negotiation.cc:769] Sending connection context
0619 20:38:50.200273 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 20:38:50.200462 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260619 20:38:50.201329 7316 heartbeater.cc:347] Connected to a master server at 127.25.62.190:46449
I20260619 20:38:50.201423 7316 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:50.201612 7316 heartbeater.cc:511] Master 127.25.62.190:46449 requested a full tablet report, sending...
I20260619 20:38:50.202102 7125 ts_manager.cc:194] Registered new tserver with Master: 611432b752984ba58352a16639bc031b (127.25.62.129:43075)
I20260619 20:38:50.202754 7125 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:59675
WARNING: no policy specified for kudu/127.25.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
I20260619 20:38:50.223666 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:46449
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:38:50.305485 7321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:50.305713 7321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:50.305750 7321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:50.306951 7321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:50.307019 7321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:50.307063 7321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:38:50.308527 7321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:46449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:50.308790 7321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:50.309023 7321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:50.313935 7329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:50.314073 7321 server_base.cc:1034] running on GCE node
W20260619 20:38:50.313971 7326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:50.314201 7327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:50.314412 7321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:50.314668 7321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:50.315811 7321 hybrid_clock.cc:648] HybridClock initialized: now 1781901530315803 us; error 28 us; skew 500 ppm
Jun 19 20:38:50 dist-test-slave-0t40 krb5kdc[7080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901530, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:50.318339 7321 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:38:50.318763 7321 webserver.cc:460] Webserver started at http://127.25.62.130:46525/ using document root <none> and password file <none>
I20260619 20:38:50.318989 7321 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:50.319056 7321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:50.319202 7321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:38:50.320252 7321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "b4fc928f8db7494baeb222eee3e33211"
format_stamp: "Formatted at 2026-06-19 20:38:50 on dist-test-slave-0t40"
server_key: "238acca44cff0c37f298d890327974ad"
server_key_iv: "52afb9e5d407145af999022c7829745a"
server_key_version: "encryptionkey@0"
I20260619 20:38:50.320617 7321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "b4fc928f8db7494baeb222eee3e33211"
format_stamp: "Formatted at 2026-06-19 20:38:50 on dist-test-slave-0t40"
server_key: "238acca44cff0c37f298d890327974ad"
server_key_iv: "52afb9e5d407145af999022c7829745a"
server_key_version: "encryptionkey@0"
I20260619 20:38:50.322018 7321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:38:50.322824 7336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:50.323035 7321 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:38:50.323122 7321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "b4fc928f8db7494baeb222eee3e33211"
format_stamp: "Formatted at 2026-06-19 20:38:50 on dist-test-slave-0t40"
server_key: "238acca44cff0c37f298d890327974ad"
server_key_iv: "52afb9e5d407145af999022c7829745a"
server_key_version: "encryptionkey@0"
I20260619 20:38:50.323197 7321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:50.334825 7321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:50.336584 7321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:50.336737 7321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:50.336966 7321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:38:50.337301 7321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:38:50.337356 7321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:50.337399 7321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:38:50.337426 7321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:50.343196 7321 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:33293
I20260619 20:38:50.343271 7449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:33293 every 8 connection(s)
I20260619 20:38:50.343607 7321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:38:50.348423 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 7321
I20260619 20:38:50.348500 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:38:50.348685 25850 external_mini_cluster.cc:1428] Setting key 09a0e68e66d5261dd8b2f2ba18535e87
Jun 19 20:38:50 dist-test-slave-0t40 krb5kdc[7080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901530, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:38:50.353204 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.344332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:45609 (local address 127.25.62.190:46449)
0619 20:38:50.344381 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:50.344383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:50.344565 (+ 182us) server_negotiation.cc:407] Connection header received
0619 20:38:50.345086 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:50.345087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:50.345099 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:50.345164 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:50.346053 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.346546 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.347535 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.347718 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.347767 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:50.350122 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:50.350124 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:50.350124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:50.350132 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:50.351619 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:50.352097 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:50.352098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:50.352099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:50.352171 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:50.352464 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:50.352465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:50.352466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:50.352625 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:50.352705 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:50.352958 (+ 253us) server_negotiation.cc:299] Negotiation successful
0619 20:38:50.353081 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:38:50.353331 7452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.344289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:45609)
0619 20:38:50.344447 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:50.344488 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:50.344964 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:50.345219 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:50.345223 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:50.345775 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:50.345972 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:50.345976 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.346645 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.346646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:50.347458 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:50.347460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.347790 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.347791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:50.347887 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:50.348403 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:50.348411 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:50.350034 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:50.351749 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:50.351757 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:50.351759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:50.352018 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:50.352273 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:50.352274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:50.352274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:50.352384 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:50.352699 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:50.352703 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:50.352817 (+ 114us) client_negotiation.cc:769] Sending connection context
0619 20:38:50.352900 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 20:38:50.353083 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260619 20:38:50.353896 7450 heartbeater.cc:347] Connected to a master server at 127.25.62.190:46449
I20260619 20:38:50.354015 7450 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:50.354231 7450 heartbeater.cc:511] Master 127.25.62.190:46449 requested a full tablet report, sending...
I20260619 20:38:50.354590 7125 ts_manager.cc:194] Registered new tserver with Master: b4fc928f8db7494baeb222eee3e33211 (127.25.62.130:33293)
I20260619 20:38:50.354982 7125 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:45609
WARNING: no policy specified for kudu/127.25.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
I20260619 20:38:50.374334 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:46449
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:38:50.453423 7455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:50.453615 7455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:50.453639 7455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:50.454797 7455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:50.454845 7455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:50.454872 7455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:38:50.456245 7455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:46449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:50.456470 7455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:50.456688 7455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:50.461500 7461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:50.461552 7460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:50.461575 7455 server_base.cc:1034] running on GCE node
W20260619 20:38:50.461683 7463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:50.461872 7455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:50.462070 7455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:50.463232 7455 hybrid_clock.cc:648] HybridClock initialized: now 1781901530463218 us; error 28 us; skew 500 ppm
Jun 19 20:38:50 dist-test-slave-0t40 krb5kdc[7080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901530, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:50.465721 7455 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:38:50.466184 7455 webserver.cc:460] Webserver started at http://127.25.62.131:44577/ using document root <none> and password file <none>
I20260619 20:38:50.466410 7455 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:50.466478 7455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:50.466625 7455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:38:50.467623 7455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "40af25e06e6f4c0ead5d34fe15bd172a"
format_stamp: "Formatted at 2026-06-19 20:38:50 on dist-test-slave-0t40"
server_key: "b2a1403bf6fd81b55a77eeda68c9c176"
server_key_iv: "6ac2a95b1f4515bb18f89b129bba9efb"
server_key_version: "encryptionkey@0"
I20260619 20:38:50.467988 7455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "40af25e06e6f4c0ead5d34fe15bd172a"
format_stamp: "Formatted at 2026-06-19 20:38:50 on dist-test-slave-0t40"
server_key: "b2a1403bf6fd81b55a77eeda68c9c176"
server_key_iv: "6ac2a95b1f4515bb18f89b129bba9efb"
server_key_version: "encryptionkey@0"
I20260619 20:38:50.469363 7455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:38:50.470109 7470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:50.470312 7455 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:38:50.470409 7455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "40af25e06e6f4c0ead5d34fe15bd172a"
format_stamp: "Formatted at 2026-06-19 20:38:50 on dist-test-slave-0t40"
server_key: "b2a1403bf6fd81b55a77eeda68c9c176"
server_key_iv: "6ac2a95b1f4515bb18f89b129bba9efb"
server_key_version: "encryptionkey@0"
I20260619 20:38:50.470489 7455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:50.479926 7455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:50.481910 7455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:50.482065 7455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:50.482299 7455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:38:50.482591 7455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:38:50.482643 7455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:50.482683 7455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:38:50.482712 7455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:50.488520 7455 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:40199
I20260619 20:38:50.488560 7583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:40199 every 8 connection(s)
I20260619 20:38:50.488906 7455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:38:50 dist-test-slave-0t40 krb5kdc[7080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901530, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:38:50.497727 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.489675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:37465 (local address 127.25.62.190:46449)
0619 20:38:50.489714 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:50.489716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:50.489956 (+ 240us) server_negotiation.cc:407] Connection header received
0619 20:38:50.490412 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:50.490413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:50.490421 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:50.490479 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:50.491358 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.491842 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.492781 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.492958 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.493010 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:50.495139 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:50.495141 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:50.495141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:50.495147 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:50.496406 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:50.496765 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:50.496766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:50.496766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:50.496824 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:50.497118 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:50.497120 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:50.497120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:50.497295 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:50.497363 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:50.497597 (+ 234us) server_negotiation.cc:299] Negotiation successful
0619 20:38:50.497665 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 20:38:50.497871 7586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.489653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:37465)
0619 20:38:50.489818 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:50.489859 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:50.490334 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:50.490539 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:50.490542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:50.491098 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:50.491285 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:50.491289 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.491916 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.491917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:50.492708 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:50.492710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.493029 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.493029 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:50.493185 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:50.493710 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:50.493718 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:50.495057 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:50.496485 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:50.496491 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:50.496492 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:50.496694 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:50.496886 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:50.496886 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:50.496887 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:50.496977 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:50.497364 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:50.497369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:50.497480 (+ 111us) client_negotiation.cc:769] Sending connection context
0619 20:38:50.497566 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 20:38:50.497681 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260619 20:38:50.498349 7584 heartbeater.cc:347] Connected to a master server at 127.25.62.190:46449
I20260619 20:38:50.498428 7584 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:50.498569 7584 heartbeater.cc:511] Master 127.25.62.190:46449 requested a full tablet report, sending...
I20260619 20:38:50.498895 7125 ts_manager.cc:194] Registered new tserver with Master: 40af25e06e6f4c0ead5d34fe15bd172a (127.25.62.131:40199)
I20260619 20:38:50.498997 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 7455
I20260619 20:38:50.499080 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:38:50.499258 25850 external_mini_cluster.cc:1428] Setting key 988b6a11dcd7ab9f705dc4f042e3eb5c
I20260619 20:38:50.499413 7125 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:37465
I20260619 20:38:50.500255 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:38:50.507326 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.501628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60670 (local address 127.25.62.190:46449)
0619 20:38:50.501697 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:50.501699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:50.501752 (+ 53us) server_negotiation.cc:407] Connection header received
0619 20:38:50.501797 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:50.501798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:50.501805 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:50.501853 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:50.502352 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.502858 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.503573 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.503777 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.503870 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:50.504974 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:50.504976 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:50.504976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:50.504983 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:50.506202 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:50.506559 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:50.506560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:50.506561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:50.506613 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:50.506816 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:50.506817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:50.506817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:50.506949 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:50.507057 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:50.507159 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 20:38:50.507236 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 20:38:50.508858 7125 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:38:50.510453 7125 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:38:50.516600 7593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.512606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:33293 (local address 127.0.0.1:48364)
0619 20:38:50.512750 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:50.512815 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:50.512883 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:50.513186 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:50.513190 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:50.513193 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:38:50.513256 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:50.513257 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.514588 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.514589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:50.515679 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:50.515681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.516311 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.516312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:50.516386 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:50.516399 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 20:38:50.516465 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 20:38:50.516513 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":24,"thread_start_us":71,"threads_started":1}
I20260619 20:38:50.516711 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.513026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53442 (local address 127.25.62.129:43075)
0619 20:38:50.513262 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:50.513267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:50.513276 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:38:50.513313 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:50.513315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:50.513332 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:50.513407 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:38:50.513604 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.514657 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.515731 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.516393 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.516458 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:50.516509 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:50.516585 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 20:38:50.516626 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":108,"threads_started":1}
I20260619 20:38:50.516777 7595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.512638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:40199 (local address 127.0.0.1:34134)
0619 20:38:50.513014 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:50.513025 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:50.513098 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:50.513467 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:50.513468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:50.513470 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:38:50.513513 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:50.513515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.514338 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.514339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:50.515593 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:50.515596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.516540 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.516541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:50.516606 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:50.516611 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:38:50.516658 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 20:38:50.516696 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":69,"threads_started":1}
I20260619 20:38:50.516935 7594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.512696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48364 (local address 127.25.62.130:33293)
0619 20:38:50.512882 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:50.512887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:50.512898 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:38:50.512958 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:50.512960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:50.512988 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:50.513089 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:38:50.513346 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.514471 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.515756 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.516226 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.516312 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:50.516377 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:50.516789 (+ 412us) server_negotiation.cc:299] Negotiation successful
0619 20:38:50.516837 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":87,"threads_started":1}
I20260619 20:38:50.517025 7596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.512884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34134 (local address 127.25.62.131:40199)
0619 20:38:50.513207 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:50.513212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:50.513220 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:38:50.513297 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:50.513299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:50.513314 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:50.513396 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:38:50.513572 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.514231 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.515678 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.516381 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.516464 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:50.516509 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:50.516877 (+ 368us) server_negotiation.cc:299] Negotiation successful
0619 20:38:50.516924 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":82,"threads_started":1}
I20260619 20:38:50.517225 7592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.512811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:43075 (local address 127.0.0.1:53442)
0619 20:38:50.513061 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:50.513069 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:50.513110 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:50.513481 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:50.513481 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:50.513483 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:38:50.513528 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:50.513529 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.514756 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.514757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:50.515622 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:50.515624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.516461 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.516462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:50.516514 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:50.516517 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:38:50.516561 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 20:38:50.516592 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":1,"thread_start_us":64,"threads_started":1}
I20260619 20:38:50.517614 7384 tablet_service.cc:1467] Processing CreateTablet for tablet ba750ef46d08411cb15da477a20e0bac (DEFAULT_TABLE table=test-table [id=1dbc33607d854cb78e9a7486b6d4c9a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:38:50.517705 7518 tablet_service.cc:1467] Processing CreateTablet for tablet ba750ef46d08411cb15da477a20e0bac (DEFAULT_TABLE table=test-table [id=1dbc33607d854cb78e9a7486b6d4c9a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:38:50.517836 7250 tablet_service.cc:1467] Processing CreateTablet for tablet ba750ef46d08411cb15da477a20e0bac (DEFAULT_TABLE table=test-table [id=1dbc33607d854cb78e9a7486b6d4c9a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:38:50.517949 7384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba750ef46d08411cb15da477a20e0bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:38:50.518025 7518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba750ef46d08411cb15da477a20e0bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:38:50.518090 7250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba750ef46d08411cb15da477a20e0bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:38:50.521235 7600 tablet_bootstrap.cc:492] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b: Bootstrap starting.
I20260619 20:38:50.521235 7599 tablet_bootstrap.cc:492] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a: Bootstrap starting.
I20260619 20:38:50.521443 7598 tablet_bootstrap.cc:492] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211: Bootstrap starting.
I20260619 20:38:50.521921 7599 tablet_bootstrap.cc:654] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a: Neither blocks nor log segments found. Creating new log.
I20260619 20:38:50.521921 7600 tablet_bootstrap.cc:654] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b: Neither blocks nor log segments found. Creating new log.
I20260619 20:38:50.522045 7598 tablet_bootstrap.cc:654] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211: Neither blocks nor log segments found. Creating new log.
I20260619 20:38:50.522583 7598 log.cc:824] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211: Log is configured to *not* fsync() on all Append() calls
I20260619 20:38:50.522612 7600 log.cc:824] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:38:50.522620 7599 log.cc:824] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a: Log is configured to *not* fsync() on all Append() calls
I20260619 20:38:50.523509 7599 tablet_bootstrap.cc:492] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a: No bootstrap required, opened a new log
I20260619 20:38:50.523566 7599 ts_tablet_manager.cc:1397] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:38:50.523711 7600 tablet_bootstrap.cc:492] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b: No bootstrap required, opened a new log
I20260619 20:38:50.523787 7600 ts_tablet_manager.cc:1397] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:38:50.525352 7599 raft_consensus.cc:348] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:50.525532 7599 raft_consensus.cc:374] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:38:50.525437 7600 raft_consensus.cc:348] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:50.525569 7599 raft_consensus.cc:729] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40af25e06e6f4c0ead5d34fe15bd172a, State: Initialized, Role: FOLLOWER
I20260619 20:38:50.525578 7600 raft_consensus.cc:374] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:38:50.525589 7598 tablet_bootstrap.cc:492] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211: No bootstrap required, opened a new log
I20260619 20:38:50.525611 7600 raft_consensus.cc:729] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611432b752984ba58352a16639bc031b, State: Initialized, Role: FOLLOWER
I20260619 20:38:50.525650 7598 ts_tablet_manager.cc:1397] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 20:38:50.525724 7600 consensus_queue.cc:260] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:50.525779 7599 consensus_queue.cc:260] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:50.526023 7600 ts_tablet_manager.cc:1428] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:38:50.526023 7316 heartbeater.cc:503] Master 127.25.62.190:46449 was elected leader, sending a full tablet report...
I20260619 20:38:50.526083 7599 ts_tablet_manager.cc:1428] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 20:38:50.526197 7584 heartbeater.cc:503] Master 127.25.62.190:46449 was elected leader, sending a full tablet report...
I20260619 20:38:50.527700 7598 raft_consensus.cc:348] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:50.527822 7598 raft_consensus.cc:374] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:38:50.527842 7598 raft_consensus.cc:729] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4fc928f8db7494baeb222eee3e33211, State: Initialized, Role: FOLLOWER
I20260619 20:38:50.527922 7598 consensus_queue.cc:260] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:50.528108 7598 ts_tablet_manager.cc:1428] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:38:50.528203 7450 heartbeater.cc:503] Master 127.25.62.190:46449 was elected leader, sending a full tablet report...
W20260619 20:38:50.594170 7451 tablet.cc:2369] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:38:50.691823 7317 tablet.cc:2369] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:38:50.739552 7585 tablet.cc:2369] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:38:50.766636 7604 raft_consensus.cc:482] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:38:50.766762 7604 raft_consensus.cc:504] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:50.767148 7604 leader_election.cc:288] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4fc928f8db7494baeb222eee3e33211 (127.25.62.130:33293), 40af25e06e6f4c0ead5d34fe15bd172a (127.25.62.131:40199)
I20260619 20:38:50.771330 7596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.767464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:52185 (local address 127.25.62.131:40199)
0619 20:38:50.767513 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:50.767516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:50.767620 (+ 104us) server_negotiation.cc:407] Connection header received
0619 20:38:50.767684 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:50.767685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:50.767695 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:50.767770 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:38:50.768023 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.768774 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.770359 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.770939 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.771028 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:50.771034 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:50.771218 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 20:38:50.771264 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:38:50.771329 7594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.767468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:45189 (local address 127.25.62.130:33293)
0619 20:38:50.767513 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:50.767516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:50.767620 (+ 104us) server_negotiation.cc:407] Connection header received
0619 20:38:50.767677 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:50.767678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:50.767694 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:50.767770 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:38:50.768034 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.768789 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.770359 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.770940 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.771024 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:50.771033 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:50.771218 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 20:38:50.771258 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:38:50.771363 7607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.767366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:33293 (local address 127.25.62.129:45189)
0619 20:38:50.767530 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:50.767542 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:50.767613 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:50.767789 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:50.767790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:50.767809 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:38:50.767907 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:50.767907 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.768881 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.768882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:50.770248 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:50.770252 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.771038 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.771039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:50.771135 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:50.771139 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:38:50.771199 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 20:38:50.771252 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":103,"thread_start_us":75,"threads_started":1}
I20260619 20:38:50.771365 7608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.767367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:40199 (local address 127.25.62.129:52185)
0619 20:38:50.767530 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:50.767542 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:50.767613 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:50.767789 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:50.767790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:50.767792 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:38:50.767905 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:50.767907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.768881 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.768882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:50.770248 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:50.770252 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.771036 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.771037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:50.771134 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:50.771140 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 20:38:50.771199 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 20:38:50.771252 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":72,"spinlock_wait_cycles":14208,"thread_start_us":105,"threads_started":1}
I20260619 20:38:50.771916 7404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba750ef46d08411cb15da477a20e0bac" candidate_uuid: "611432b752984ba58352a16639bc031b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc928f8db7494baeb222eee3e33211" is_pre_election: true
I20260619 20:38:50.772008 7538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba750ef46d08411cb15da477a20e0bac" candidate_uuid: "611432b752984ba58352a16639bc031b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" is_pre_election: true
I20260619 20:38:50.772096 7404 raft_consensus.cc:2454] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611432b752984ba58352a16639bc031b in term 0.
I20260619 20:38:50.772119 7538 raft_consensus.cc:2454] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 611432b752984ba58352a16639bc031b in term 0.
I20260619 20:38:50.772329 7203 leader_election.cc:302] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 611432b752984ba58352a16639bc031b, b4fc928f8db7494baeb222eee3e33211; no voters:
I20260619 20:38:50.772502 7604 raft_consensus.cc:2781] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:38:50.772547 7604 raft_consensus.cc:482] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:38:50.772563 7604 raft_consensus.cc:3037] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:38:50.773384 7604 raft_consensus.cc:504] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:50.773550 7604 leader_election.cc:288] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 1 election: Requested vote from peers b4fc928f8db7494baeb222eee3e33211 (127.25.62.130:33293), 40af25e06e6f4c0ead5d34fe15bd172a (127.25.62.131:40199)
I20260619 20:38:50.773710 7538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba750ef46d08411cb15da477a20e0bac" candidate_uuid: "611432b752984ba58352a16639bc031b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40af25e06e6f4c0ead5d34fe15bd172a"
I20260619 20:38:50.773715 7404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba750ef46d08411cb15da477a20e0bac" candidate_uuid: "611432b752984ba58352a16639bc031b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc928f8db7494baeb222eee3e33211"
I20260619 20:38:50.773794 7538 raft_consensus.cc:3037] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:38:50.773819 7404 raft_consensus.cc:3037] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:38:50.774531 7538 raft_consensus.cc:2454] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611432b752984ba58352a16639bc031b in term 1.
I20260619 20:38:50.774627 7404 raft_consensus.cc:2454] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 611432b752984ba58352a16639bc031b in term 1.
I20260619 20:38:50.774701 7206 leader_election.cc:302] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40af25e06e6f4c0ead5d34fe15bd172a, 611432b752984ba58352a16639bc031b; no voters:
I20260619 20:38:50.774813 7604 raft_consensus.cc:2781] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:38:50.774976 7604 raft_consensus.cc:686] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [term 1 LEADER]: Becoming Leader. State: Replica: 611432b752984ba58352a16639bc031b, State: Running, Role: LEADER
I20260619 20:38:50.775085 7604 consensus_queue.cc:237] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:50.775954 7125 catalog_manager.cc:5495] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b reported cstate change: term changed from 0 to 1, leader changed from <none> to 611432b752984ba58352a16639bc031b (127.25.62.129). New cstate: current_term: 1 leader_uuid: "611432b752984ba58352a16639bc031b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } health_report { overall_health: HEALTHY } } }
I20260619 20:38:50.797621 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.795363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53456 (local address 127.25.62.129:43075)
0619 20:38:50.795432 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:50.795434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:50.795442 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:38:50.795475 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:50.795476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:50.795494 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:50.795558 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:38:50.795669 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.796121 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.796861 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:50.797103 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:50.797157 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:50.797349 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:38:50.797449 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:38:50.797512 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:50.797549 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 20:38:50.797565 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:38:50.799335 7538 raft_consensus.cc:1261] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a [term 1 FOLLOWER]: Refusing update from remote peer 611432b752984ba58352a16639bc031b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:38:50.799433 7404 raft_consensus.cc:1261] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211 [term 1 FOLLOWER]: Refusing update from remote peer 611432b752984ba58352a16639bc031b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:38:50.799685 7609 consensus_queue.cc:1035] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [LEADER]: Connected to new peer: Peer: permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:38:50.799808 7604 consensus_queue.cc:1035] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:38:50.802135 7612 mvcc.cc:205] Tried to move back new op lower bound from 7298668670151544832 to 7298668670055407616. Current Snapshot: MvccSnapshot[applied={T|T < 7298668670151544832}]
I20260619 20:38:50.802381 7614 mvcc.cc:205] Tried to move back new op lower bound from 7298668670151544832 to 7298668670055407616. Current Snapshot: MvccSnapshot[applied={T|T < 7298668670151544832}]
I20260619 20:38:50.804883 7124 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:38:50.804989 7124 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:38:50.805776 7124 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477: Sending DeleteTablet for 3 replicas of tablet ba750ef46d08411cb15da477a20e0bac
I20260619 20:38:50.806095 7384 tablet_service.cc:1514] Processing DeleteTablet for tablet ba750ef46d08411cb15da477a20e0bac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:38:50 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:48364
I20260619 20:38:50.806106 7518 tablet_service.cc:1514] Processing DeleteTablet for tablet ba750ef46d08411cb15da477a20e0bac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:38:50 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:34134
I20260619 20:38:50.806136 7250 tablet_service.cc:1514] Processing DeleteTablet for tablet ba750ef46d08411cb15da477a20e0bac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:38:50 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:53442
I20260619 20:38:50.806339 7622 tablet_replica.cc:330] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a: stopping tablet replica
I20260619 20:38:50.806375 7621 tablet_replica.cc:330] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211: stopping tablet replica
I20260619 20:38:50.806404 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 7187
I20260619 20:38:50.806447 7622 raft_consensus.cc:2229] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:38:50.806527 7621 raft_consensus.cc:2229] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:38:50.806552 7622 raft_consensus.cc:2258] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:38:50.806654 7621 raft_consensus.cc:2258] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:38:50.807996 7621 ts_tablet_manager.cc:1905] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:38:50.809938 7621 ts_tablet_manager.cc:1918] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:38:50.810230 7621 log.cc:1196] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/ba750ef46d08411cb15da477a20e0bac
I20260619 20:38:50.810528 7621 ts_tablet_manager.cc:1939] T ba750ef46d08411cb15da477a20e0bac P b4fc928f8db7494baeb222eee3e33211: Deleting consensus metadata
I20260619 20:38:50.810951 7109 catalog_manager.cc:4841] TS b4fc928f8db7494baeb222eee3e33211 (127.25.62.130:33293): tablet ba750ef46d08411cb15da477a20e0bac (table test-table [id=1dbc33607d854cb78e9a7486b6d4c9a0]) successfully deleted
W20260619 20:38:50.812158 7109 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.62.129:43075 (error 108)
I20260619 20:38:50.812462 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 7321
I20260619 20:38:50.812634 7622 ts_tablet_manager.cc:1905] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:38:50.814577 7622 ts_tablet_manager.cc:1918] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:38:50.814654 7622 log.cc:1196] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/ba750ef46d08411cb15da477a20e0bac
I20260619 20:38:50.814921 7622 ts_tablet_manager.cc:1939] T ba750ef46d08411cb15da477a20e0bac P 40af25e06e6f4c0ead5d34fe15bd172a: Deleting consensus metadata
I20260619 20:38:50.815442 7112 catalog_manager.cc:4841] TS 40af25e06e6f4c0ead5d34fe15bd172a (127.25.62.131:40199): tablet ba750ef46d08411cb15da477a20e0bac (table test-table [id=1dbc33607d854cb78e9a7486b6d4c9a0]) successfully deleted
I20260619 20:38:50.815887 7592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:50.815752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:43075 (local address 127.0.0.1:53458)
0619 20:38:50.815815 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:50.815859 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.129:43075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 20:38:50.816016 7109 catalog_manager.cc:4584] TS 611432b752984ba58352a16639bc031b (127.25.62.129:43075): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ba750ef46d08411cb15da477a20e0bac: Network error: Client connection negotiation failed: client connection to 127.25.62.129:43075: connect: Connection refused (error 111)
I20260619 20:38:50.818849 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 7455
I20260619 20:38:50.823462 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 7093
I20260619 20:38:50.828971 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:46449
--webserver_interface=127.25.62.190
--webserver_port=41215
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:46449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:38:50.910526 7625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:50.910777 7625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:50.910813 7625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:50.911962 7625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:38:50.912032 7625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:50.912057 7625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:50.912081 7625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:38:50.912109 7625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:38:50.913609 7625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:46449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:46449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=41215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:50.913890 7625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:50.914129 7625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:50.922842 7632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:50.922844 7625 server_base.cc:1034] running on GCE node
W20260619 20:38:50.922847 7634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:50.922857 7631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:50.923339 7625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:50.923678 7625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:50.924842 7625 hybrid_clock.cc:648] HybridClock initialized: now 1781901530924832 us; error 43 us; skew 500 ppm
Jun 19 20:38:50 dist-test-slave-0t40 krb5kdc[7080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901530, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:50.931917 7625 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:38:50.932363 7625 webserver.cc:460] Webserver started at http://127.25.62.190:41215/ using document root <none> and password file <none>
I20260619 20:38:50.932616 7625 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:50.932700 7625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:50.934154 7625 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:38:50.934834 7641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:50.935034 7625 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:38:50.935140 7625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "0bd0501973194ca1b6934e64e4e18477"
format_stamp: "Formatted at 2026-06-19 20:38:49 on dist-test-slave-0t40"
server_key: "66802977e17c225f20b886a8c457dabd"
server_key_iv: "d0320f6cd9752cb4f15e1ed4f1f0108d"
server_key_version: "encryptionkey@0"
I20260619 20:38:50.935549 7625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:50.951642 7625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:50.953868 7625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:50.954032 7625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:50.957919 7625 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:46449
I20260619 20:38:50.957952 7693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:46449 every 8 connection(s)
I20260619 20:38:50.958336 7625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:38:50.959860 7694 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:38:50.960502 7694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477: Bootstrap starting.
I20260619 20:38:50.963557 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 7625
I20260619 20:38:50.963835 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:43075
--local_ip_for_outbound_sockets=127.25.62.129
--tserver_master_addrs=127.25.62.190:46449
--webserver_port=43001
--webserver_interface=127.25.62.129
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:38:50.969506 7694 log.cc:824] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477: Log is configured to *not* fsync() on all Append() calls
I20260619 20:38:50.971396 7694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:38:50.971688 7694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477: Bootstrap complete.
I20260619 20:38:50.974107 7694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd0501973194ca1b6934e64e4e18477" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46449 } }
I20260619 20:38:50.974347 7694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0501973194ca1b6934e64e4e18477, State: Initialized, Role: FOLLOWER
I20260619 20:38:50.974566 7694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd0501973194ca1b6934e64e4e18477" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46449 } }
I20260619 20:38:50.974663 7694 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:38:50.974720 7694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:38:50.974766 7694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 1 FOLLOWER]: Advancing to term 2
I20260619 20:38:50.975768 7694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd0501973194ca1b6934e64e4e18477" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46449 } }
I20260619 20:38:50.975893 7694 leader_election.cc:302] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bd0501973194ca1b6934e64e4e18477; no voters:
I20260619 20:38:50.976068 7694 leader_election.cc:288] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 20:38:50.976136 7698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 20:38:50.976286 7698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 2 LEADER]: Becoming Leader. State: Replica: 0bd0501973194ca1b6934e64e4e18477, State: Running, Role: LEADER
I20260619 20:38:50.976370 7694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:38:50.976384 7698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd0501973194ca1b6934e64e4e18477" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46449 } }
I20260619 20:38:50.976666 7698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bd0501973194ca1b6934e64e4e18477. Latest consensus state: current_term: 2 leader_uuid: "0bd0501973194ca1b6934e64e4e18477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd0501973194ca1b6934e64e4e18477" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46449 } } }
I20260619 20:38:50.976727 7698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [sys.catalog]: This master's current role is: LEADER
I20260619 20:38:50.976905 7699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0bd0501973194ca1b6934e64e4e18477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd0501973194ca1b6934e64e4e18477" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46449 } } }
I20260619 20:38:50.976959 7699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [sys.catalog]: This master's current role is: LEADER
I20260619 20:38:50.977406 7702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:38:50.978156 7702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:38:50.978252 7702 catalog_manager.cc:1259] Loaded cluster ID: 571a2301f0604d31afe00008dc4b5c88
I20260619 20:38:50.978284 7702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:38:50.978515 7702 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 20:38:50.983796 7702 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 20:38:50.986758 7702 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:38:50.986954 7702 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477: Loaded TSK: 0
I20260619 20:38:50.987155 7702 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 20:38:51.048342 7696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:51.048532 7696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:51.048554 7696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:51.049747 7696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:51.049787 7696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:51.049813 7696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:38:51.051218 7696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:43075
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=43001
--tserver_master_addrs=127.25.62.190:46449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:51.051450 7696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:51.051689 7696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:51.056591 7722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:51.056603 7721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:51.056674 7724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:51.056871 7696 server_base.cc:1034] running on GCE node
I20260619 20:38:51.057139 7696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:51.057391 7696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:51.058538 7696 hybrid_clock.cc:648] HybridClock initialized: now 1781901531058527 us; error 26 us; skew 500 ppm
Jun 19 20:38:51 dist-test-slave-0t40 krb5kdc[7080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901531, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:51.060913 7696 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:38:51.061415 7696 webserver.cc:460] Webserver started at http://127.25.62.129:43001/ using document root <none> and password file <none>
I20260619 20:38:51.061664 7696 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:51.061719 7696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:51.063005 7696 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:38:51.063676 7731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:51.063896 7696 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:38:51.063997 7696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "611432b752984ba58352a16639bc031b"
format_stamp: "Formatted at 2026-06-19 20:38:50 on dist-test-slave-0t40"
server_key: "2e0797f7a91a232aed591f9d8cf2398e"
server_key_iv: "aa1f652056722269aa80a9a560849191"
server_key_version: "encryptionkey@0"
I20260619 20:38:51.064323 7696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:51.086136 7696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:51.088260 7696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:51.088435 7696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:51.088673 7696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:38:51.089246 7738 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 20:38:51.091074 7696 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 20:38:51.091141 7696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 20:38:51.091190 7696 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 20:38:51.091789 7696 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 20:38:51.091841 7696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 20:38:51.091915 7738 tablet_bootstrap.cc:492] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b: Bootstrap starting.
I20260619 20:38:51.097136 7696 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:43075
I20260619 20:38:51.097625 7696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:38:51.098042 7845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:43075 every 8 connection(s)
I20260619 20:38:51.099179 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 7696
I20260619 20:38:51.099511 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:33293
--local_ip_for_outbound_sockets=127.25.62.130
--tserver_master_addrs=127.25.62.190:46449
--webserver_port=46525
--webserver_interface=127.25.62.130
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:38:51.103693 7738 log.cc:824] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:38:51.106230 7738 tablet_bootstrap.cc:492] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:38:51.106527 7738 tablet_bootstrap.cc:492] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b: Bootstrap complete.
I20260619 20:38:51.108044 7738 ts_tablet_manager.cc:1397] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.004s
Jun 19 20:38:51 dist-test-slave-0t40 krb5kdc[7080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901531, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:38:51.109331 7738 raft_consensus.cc:348] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:51.109597 7738 raft_consensus.cc:729] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611432b752984ba58352a16639bc031b, State: Initialized, Role: FOLLOWER
I20260619 20:38:51.109752 7738 consensus_queue.cc:260] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:51.110116 7738 ts_tablet_manager.cc:1428] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:38:51.113556 7850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.100903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:42413 (local address 127.25.62.190:46449)
0619 20:38:51.101153 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:51.101161 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:51.101200 (+ 39us) server_negotiation.cc:407] Connection header received
0619 20:38:51.102086 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:51.102094 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:51.102156 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:51.102283 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:51.103046 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.104509 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.106260 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.106928 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.107045 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:51.110172 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:51.110176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:51.110176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:51.110188 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:51.111911 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:51.112388 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:51.112390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:51.112390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:51.112475 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:51.112729 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:51.112729 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:51.112729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:51.112939 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:51.112996 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:51.113316 (+ 320us) server_negotiation.cc:299] Negotiation successful
0619 20:38:51.113420 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":68,"threads_started":1}
I20260619 20:38:51.113629 7848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.100436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:42413)
0619 20:38:51.100942 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:51.101000 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:51.101693 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:51.102476 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:51.102479 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:51.102739 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:51.102927 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:51.102931 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.104654 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.104655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:51.106148 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:51.106152 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.107165 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.107166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:51.107318 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:51.107958 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:51.107975 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:51.110020 (+ 2045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:51.112004 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:51.112011 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:51.112014 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:51.112300 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:51.112539 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:51.112540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:51.112540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:51.112662 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:51.113004 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:51.113009 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:51.113175 (+ 166us) client_negotiation.cc:769] Sending connection context
0619 20:38:51.113273 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 20:38:51.113442 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":101,"threads_started":1}
I20260619 20:38:51.114441 7846 heartbeater.cc:347] Connected to a master server at 127.25.62.190:46449
I20260619 20:38:51.114560 7846 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:51.114738 7846 heartbeater.cc:511] Master 127.25.62.190:46449 requested a full tablet report, sending...
I20260619 20:38:51.115376 7658 ts_manager.cc:194] Registered new tserver with Master: 611432b752984ba58352a16639bc031b (127.25.62.129:43075)
I20260619 20:38:51.116496 7658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:42413
I20260619 20:38:51.117369 7846 heartbeater.cc:503] Master 127.25.62.190:46449 was elected leader, sending a full tablet report...
Jun 19 20:38:51 dist-test-slave-0t40 krb5kdc[7080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901530, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for kudu/127.25.62.129@KRBTEST.COM
I20260619 20:38:51.124066 7853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.115917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:43075 (local address 127.0.0.1:53464)
0619 20:38:51.116086 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:51.116126 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:51.116194 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:51.116380 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:51.116384 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:51.116695 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:51.116751 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:51.116753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.118065 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.118065 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:51.118608 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:51.118610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.119232 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.119233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:51.119285 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:51.119747 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:51.119752 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:51.121237 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:51.123015 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:51.123021 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:51.123022 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:51.123201 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:51.123409 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:51.123410 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:51.123410 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:51.123459 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:51.123767 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:51.123772 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:51.123818 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 20:38:51.123897 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 20:38:51.124003 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260619 20:38:51.124125 7854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.116104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53464 (local address 127.25.62.129:43075)
0619 20:38:51.116238 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:51.116242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:51.116251 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:38:51.116283 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:51.116284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:51.116296 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:51.116365 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:51.116844 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.117985 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.118691 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.119163 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.119233 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:51.121314 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:51.121321 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:51.121322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:51.121328 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:51.122927 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:51.123281 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:51.123284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:51.123284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:51.123345 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:51.123516 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:51.123517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:51.123517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:51.123674 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:51.123749 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:51.123950 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 20:38:51.124041 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260619 20:38:51.124518 7780 tablet_service.cc:1514] Processing DeleteTablet for tablet ba750ef46d08411cb15da477a20e0bac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:38:50 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:53464
I20260619 20:38:51.125144 7855 tablet_replica.cc:330] stopping tablet replica
I20260619 20:38:51.125244 7855 raft_consensus.cc:2229] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:38:51.125311 7855 raft_consensus.cc:2258] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:38:51.125599 7855 ts_tablet_manager.cc:1905] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:38:51.127672 7855 ts_tablet_manager.cc:1918] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:38:51.127733 7855 log.cc:1196] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/wal/wals/ba750ef46d08411cb15da477a20e0bac
I20260619 20:38:51.127954 7855 ts_tablet_manager.cc:1939] T ba750ef46d08411cb15da477a20e0bac P 611432b752984ba58352a16639bc031b: Deleting consensus metadata
I20260619 20:38:51.128432 7642 catalog_manager.cc:4841] TS 611432b752984ba58352a16639bc031b (127.25.62.129:43075): tablet ba750ef46d08411cb15da477a20e0bac (table test-table [id=1dbc33607d854cb78e9a7486b6d4c9a0]) successfully deleted
W20260619 20:38:51.186944 7849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:51.187163 7849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:51.187203 7849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:51.188333 7849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:51.188395 7849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:51.188441 7849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:38:51.189927 7849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:33293
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=46525
--tserver_master_addrs=127.25.62.190:46449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:51.190184 7849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:51.190404 7849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:38:51.195340 7849 server_base.cc:1034] running on GCE node
W20260619 20:38:51.195345 7860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:51.195453 7861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:51.195348 7863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:51.195814 7849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:51.196058 7849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:51.197222 7849 hybrid_clock.cc:648] HybridClock initialized: now 1781901531197212 us; error 30 us; skew 500 ppm
Jun 19 20:38:51 dist-test-slave-0t40 krb5kdc[7080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901531, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:51.199666 7849 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:38:51.200109 7849 webserver.cc:460] Webserver started at http://127.25.62.130:46525/ using document root <none> and password file <none>
I20260619 20:38:51.200335 7849 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:51.200414 7849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:51.201807 7849 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:38:51.202415 7870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:51.202621 7849 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:38:51.202713 7849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "b4fc928f8db7494baeb222eee3e33211"
format_stamp: "Formatted at 2026-06-19 20:38:50 on dist-test-slave-0t40"
server_key: "238acca44cff0c37f298d890327974ad"
server_key_iv: "52afb9e5d407145af999022c7829745a"
server_key_version: "encryptionkey@0"
I20260619 20:38:51.203042 7849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:51.224366 7849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:51.226697 7849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:51.226837 7849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:51.227077 7849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:38:51.227391 7849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:38:51.227443 7849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:51.227483 7849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:38:51.227511 7849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:51.233136 7849 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:33293
I20260619 20:38:51.233204 7983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:33293 every 8 connection(s)
I20260619 20:38:51.233546 7849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:38:51.236331 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 7849
I20260619 20:38:51.236616 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:40199
--local_ip_for_outbound_sockets=127.25.62.131
--tserver_master_addrs=127.25.62.190:46449
--webserver_port=44577
--webserver_interface=127.25.62.131
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 20:38:51 dist-test-slave-0t40 krb5kdc[7080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901531, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:38:51.244248 7850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.234257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:50473 (local address 127.25.62.190:46449)
0619 20:38:51.234334 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:51.234337 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:51.234561 (+ 224us) server_negotiation.cc:407] Connection header received
0619 20:38:51.235045 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:51.235046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:51.235056 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:51.235109 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:51.236004 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.236568 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.238231 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.238630 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.238710 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:51.241146 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:51.241147 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:51.241147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:51.241155 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:51.242677 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:51.243189 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:51.243189 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:51.243190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:51.243257 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:51.243568 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:51.243569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:51.243570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:51.243688 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:51.243754 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:51.244110 (+ 356us) server_negotiation.cc:299] Negotiation successful
0619 20:38:51.244186 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 20:38:51.244549 7986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.234248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:50473)
0619 20:38:51.234432 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:51.234468 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:51.234957 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:51.235163 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:51.235166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:51.235726 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:51.235920 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:51.235925 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.236660 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.236661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:51.238090 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:51.238094 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.238716 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.238717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:51.238828 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:51.239327 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:51.239337 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:51.241018 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:51.242766 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:51.242773 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:51.242776 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:51.243098 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:51.243324 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:51.243324 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:51.243325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:51.243473 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:51.243768 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:51.243774 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:51.243953 (+ 179us) client_negotiation.cc:769] Sending connection context
0619 20:38:51.244081 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 20:38:51.244277 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":106,"threads_started":1}
I20260619 20:38:51.245090 7984 heartbeater.cc:347] Connected to a master server at 127.25.62.190:46449
I20260619 20:38:51.245194 7984 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:51.245404 7984 heartbeater.cc:511] Master 127.25.62.190:46449 requested a full tablet report, sending...
I20260619 20:38:51.245811 7658 ts_manager.cc:194] Registered new tserver with Master: b4fc928f8db7494baeb222eee3e33211 (127.25.62.130:33293)
I20260619 20:38:51.246170 7658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:50473
W20260619 20:38:51.319160 7987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:51.319363 7987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:51.319387 7987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:51.320549 7987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:51.320593 7987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:51.320621 7987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:38:51.322229 7987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:40199
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=44577
--tserver_master_addrs=127.25.62.190:46449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:51.322466 7987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:51.322690 7987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:51.327838 7995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:51.327919 7992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:51.327960 7993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:51.328025 7987 server_base.cc:1034] running on GCE node
I20260619 20:38:51.328289 7987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:51.328543 7987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:51.329725 7987 hybrid_clock.cc:648] HybridClock initialized: now 1781901531329698 us; error 41 us; skew 500 ppm
Jun 19 20:38:51 dist-test-slave-0t40 krb5kdc[7080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901531, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:51.332783 7987 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:38:51.333494 7987 webserver.cc:460] Webserver started at http://127.25.62.131:44577/ using document root <none> and password file <none>
I20260619 20:38:51.333765 7987 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:51.333858 7987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:51.335337 7987 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:38:51.336025 8002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:51.336232 7987 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:38:51.336347 7987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "40af25e06e6f4c0ead5d34fe15bd172a"
format_stamp: "Formatted at 2026-06-19 20:38:50 on dist-test-slave-0t40"
server_key: "b2a1403bf6fd81b55a77eeda68c9c176"
server_key_iv: "6ac2a95b1f4515bb18f89b129bba9efb"
server_key_version: "encryptionkey@0"
I20260619 20:38:51.336735 7987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:51.345177 7987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:51.347582 7987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:51.347757 7987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:51.347998 7987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:38:51.348310 7987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:38:51.348367 7987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:51.348414 7987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:38:51.348443 7987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:51.354868 7987 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:40199
I20260619 20:38:51.354902 8115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:40199 every 8 connection(s)
I20260619 20:38:51.355263 7987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:38:51 dist-test-slave-0t40 krb5kdc[7080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901531, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:38:51.362310 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 7987
I20260619 20:38:51.364619 7850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.356076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:42279 (local address 127.25.62.190:46449)
0619 20:38:51.356155 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:51.356157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:51.356391 (+ 234us) server_negotiation.cc:407] Connection header received
0619 20:38:51.356897 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:51.356899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:51.356909 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:51.356968 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:51.357949 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.358449 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.359407 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.359584 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.359632 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:51.361669 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:51.361670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:51.361670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:51.361678 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:51.363267 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:51.363735 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:51.363736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:51.363736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:51.363799 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:51.364033 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:51.364034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:51.364034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:51.364158 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:51.364223 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:51.364494 (+ 271us) server_negotiation.cc:299] Negotiation successful
0619 20:38:51.364558 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 20:38:51.364832 8118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.356059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:42279)
0619 20:38:51.356268 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:51.356305 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:51.356794 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:51.357102 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:51.357107 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:51.357659 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:51.357864 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:51.357869 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.358534 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.358535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:51.359336 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:51.359339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.359651 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.359651 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:51.359747 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:51.360182 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:51.360190 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:51.361565 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:51.363352 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:51.363358 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:51.363360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:51.363658 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:51.363875 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:51.363876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:51.363876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:51.363969 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:51.364219 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:51.364223 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:51.364339 (+ 116us) client_negotiation.cc:769] Sending connection context
0619 20:38:51.364469 (+ 130us) client_negotiation.cc:240] Negotiation successful
0619 20:38:51.364620 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":110,"threads_started":1}
I20260619 20:38:51.365332 8116 heartbeater.cc:347] Connected to a master server at 127.25.62.190:46449
I20260619 20:38:51.365440 8116 heartbeater.cc:464] Registering TS with master...
I20260619 20:38:51.365628 8116 heartbeater.cc:511] Master 127.25.62.190:46449 requested a full tablet report, sending...
I20260619 20:38:51.366015 7658 ts_manager.cc:194] Registered new tserver with Master: 40af25e06e6f4c0ead5d34fe15bd172a (127.25.62.131:40199)
I20260619 20:38:51.366497 7658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:42279
I20260619 20:38:51.369529 8120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.362683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60686 (local address 127.25.62.190:46449)
0619 20:38:51.362790 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:51.362792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:51.362877 (+ 85us) server_negotiation.cc:407] Connection header received
0619 20:38:51.362919 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:51.362920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:51.362935 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:51.363001 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:51.363603 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.364323 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.365372 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.365659 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.365738 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:51.367186 (+ 1448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:51.367188 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:51.367188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:51.367195 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:51.368387 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:51.368728 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:51.368729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:51.368729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:51.368792 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:51.368989 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:51.368990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:51.368990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:51.369126 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:51.369319 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:51.369356 (+ 37us) server_negotiation.cc:299] Negotiation successful
0619 20:38:51.369431 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":54,"threads_started":1}
I20260619 20:38:51.370198 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:38:51.372721 8120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.370543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60690 (local address 127.25.62.190:46449)
0619 20:38:51.370579 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:51.370581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:51.370624 (+ 43us) server_negotiation.cc:407] Connection header received
0619 20:38:51.370680 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:51.370681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:51.370705 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:51.370768 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:38:51.370856 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.371394 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.372141 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.372327 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.372392 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:51.372487 (+ 95us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:38:51.372578 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:38:51.372647 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:51.372661 (+ 14us) server_negotiation.cc:299] Negotiation successful
0619 20:38:51.372677 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:38:51.373103 7658 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:60690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:38:51.374643 7658 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:38:51.376863 7780 tablet_service.cc:1467] Processing CreateTablet for tablet 3e2fa55b13ba41e388dbd89fbce7ca8f (DEFAULT_TABLE table=test-table [id=545e359c87ab4c81baa878f28c6b445a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:38:51.377103 7780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e2fa55b13ba41e388dbd89fbce7ca8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:38:51.378598 7738 tablet_bootstrap.cc:492] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b: Bootstrap starting.
I20260619 20:38:51.378901 7738 tablet_bootstrap.cc:654] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b: Neither blocks nor log segments found. Creating new log.
I20260619 20:38:51.379856 7738 tablet_bootstrap.cc:492] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b: No bootstrap required, opened a new log
I20260619 20:38:51.379897 7738 ts_tablet_manager.cc:1397] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 20:38:51.380048 7738 raft_consensus.cc:348] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:51.380121 7738 raft_consensus.cc:374] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:38:51.380169 7738 raft_consensus.cc:729] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611432b752984ba58352a16639bc031b, State: Initialized, Role: FOLLOWER
I20260619 20:38:51.380220 7738 consensus_queue.cc:260] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:51.380316 7738 ts_tablet_manager.cc:1428] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 20:38:51.380327 7853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.376471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:33293 (local address 127.0.0.1:48374)
0619 20:38:51.376588 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:51.376599 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:51.376630 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:51.376896 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:51.376897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:51.376898 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:38:51.376968 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:51.376970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.378110 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.378112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:51.379076 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:51.379080 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.380064 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.380065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:51.380138 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:51.380162 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 20:38:51.380214 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 20:38:51.380261 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260619 20:38:51.380688 8122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.376550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48374 (local address 127.25.62.130:33293)
0619 20:38:51.376731 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:51.376735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:51.376742 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:38:51.376774 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:51.376776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:51.376788 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:51.376955 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:38:51.377028 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.378001 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.379165 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.379977 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.380050 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:51.380113 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:51.380250 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 20:38:51.380608 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":98,"threads_started":1}
I20260619 20:38:51.381496 7918 tablet_service.cc:1467] Processing CreateTablet for tablet 3e2fa55b13ba41e388dbd89fbce7ca8f (DEFAULT_TABLE table=test-table [id=545e359c87ab4c81baa878f28c6b445a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:38:51.381858 7918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e2fa55b13ba41e388dbd89fbce7ca8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:38:51.382575 8123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.376558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34146 (local address 127.25.62.131:40199)
0619 20:38:51.377143 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:51.377147 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:51.378818 (+ 1671us) server_negotiation.cc:407] Connection header received
0619 20:38:51.378888 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:51.378891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:51.378913 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:51.378984 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:38:51.379196 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.380293 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.381546 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.382254 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.382311 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:51.382345 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:51.382487 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 20:38:51.382516 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":94,"threads_started":1}
I20260619 20:38:51.382578 8124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.376479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:40199 (local address 127.0.0.1:34146)
0619 20:38:51.378755 (+ 2276us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:51.378765 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:51.378821 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:51.379062 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:51.379063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:51.379065 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:38:51.379105 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:51.379107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.380404 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.380405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:51.381465 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:51.381468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.382338 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.382339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:51.382401 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:51.382405 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:38:51.382469 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 20:38:51.382508 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2260,"mutex_wait_us":2111,"thread_start_us":101,"threads_started":1}
I20260619 20:38:51.384641 8126 tablet_bootstrap.cc:492] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211: Bootstrap starting.
I20260619 20:38:51.385195 8050 tablet_service.cc:1467] Processing CreateTablet for tablet 3e2fa55b13ba41e388dbd89fbce7ca8f (DEFAULT_TABLE table=test-table [id=545e359c87ab4c81baa878f28c6b445a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:38:51.385335 8126 tablet_bootstrap.cc:654] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211: Neither blocks nor log segments found. Creating new log.
I20260619 20:38:51.385514 8050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e2fa55b13ba41e388dbd89fbce7ca8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:38:51.385864 8126 log.cc:824] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211: Log is configured to *not* fsync() on all Append() calls
I20260619 20:38:51.386508 8126 tablet_bootstrap.cc:492] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211: No bootstrap required, opened a new log
I20260619 20:38:51.386561 8126 ts_tablet_manager.cc:1397] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:38:51.388063 8128 tablet_bootstrap.cc:492] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a: Bootstrap starting.
I20260619 20:38:51.387861 8126 raft_consensus.cc:348] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:51.388154 8126 raft_consensus.cc:374] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:38:51.388209 8126 raft_consensus.cc:729] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4fc928f8db7494baeb222eee3e33211, State: Initialized, Role: FOLLOWER
I20260619 20:38:51.388387 8126 consensus_queue.cc:260] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:51.388662 8126 ts_tablet_manager.cc:1428] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:38:51.388681 7984 heartbeater.cc:503] Master 127.25.62.190:46449 was elected leader, sending a full tablet report...
I20260619 20:38:51.388731 8128 tablet_bootstrap.cc:654] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a: Neither blocks nor log segments found. Creating new log.
I20260619 20:38:51.389356 8128 log.cc:824] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a: Log is configured to *not* fsync() on all Append() calls
I20260619 20:38:51.390057 8128 tablet_bootstrap.cc:492] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a: No bootstrap required, opened a new log
I20260619 20:38:51.390110 8128 ts_tablet_manager.cc:1397] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:38:51.391342 8128 raft_consensus.cc:348] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:51.391469 8128 raft_consensus.cc:374] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:38:51.391489 8128 raft_consensus.cc:729] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40af25e06e6f4c0ead5d34fe15bd172a, State: Initialized, Role: FOLLOWER
I20260619 20:38:51.391563 8128 consensus_queue.cc:260] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:51.391827 8116 heartbeater.cc:503] Master 127.25.62.190:46449 was elected leader, sending a full tablet report...
I20260619 20:38:51.391834 8128 ts_tablet_manager.cc:1428] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260619 20:38:51.484076 7985 tablet.cc:2369] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:38:51.564340 8131 raft_consensus.cc:482] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:38:51.564469 8131 raft_consensus.cc:504] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:51.564909 8131 leader_election.cc:288] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4fc928f8db7494baeb222eee3e33211 (127.25.62.130:33293), 611432b752984ba58352a16639bc031b (127.25.62.129:43075)
I20260619 20:38:51.567812 8118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.565137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:33293 (local address 127.25.62.131:39205)
0619 20:38:51.565196 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:51.565210 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:51.565286 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:51.565438 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:51.565439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:51.565441 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:38:51.565507 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:51.565509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.566155 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.566155 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:51.566970 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:51.566972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.567552 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.567553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:51.567634 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:51.567637 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:38:51.567702 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 20:38:51.567742 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 20:38:51.567819 8122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.565192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:39205 (local address 127.25.62.130:33293)
0619 20:38:51.565265 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:51.565268 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:51.565280 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:38:51.565351 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:51.565352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:51.565362 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:51.565421 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:38:51.565597 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.566071 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.567070 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.567458 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.567553 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:51.567562 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:51.567707 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 20:38:51.567742 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 20:38:51.568111 7938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e2fa55b13ba41e388dbd89fbce7ca8f" candidate_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc928f8db7494baeb222eee3e33211" is_pre_election: true
I20260619 20:38:51.568280 7938 raft_consensus.cc:2454] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40af25e06e6f4c0ead5d34fe15bd172a in term 0.
I20260619 20:38:51.568425 8132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.565279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:43075 (local address 127.25.62.131:55565)
0619 20:38:51.565441 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:51.565454 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:51.565515 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:51.565681 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:51.565681 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:51.565683 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:38:51.565754 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:51.565757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.566409 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.566411 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:51.567238 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:51.567244 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.568159 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.568160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:51.568232 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:51.568237 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:38:51.568294 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 20:38:51.568340 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":12,"thread_start_us":102,"threads_started":1}
I20260619 20:38:51.568490 7854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.565347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:55565 (local address 127.25.62.129:43075)
0619 20:38:51.565393 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:51.565395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:51.565505 (+ 110us) server_negotiation.cc:407] Connection header received
0619 20:38:51.565568 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:51.565569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:51.565595 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:51.565676 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:38:51.565831 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.566324 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.567340 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.567862 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.567944 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:51.568028 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:51.568376 (+ 348us) server_negotiation.cc:299] Negotiation successful
0619 20:38:51.568422 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:38:51.568514 8003 leader_election.cc:302] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40af25e06e6f4c0ead5d34fe15bd172a, b4fc928f8db7494baeb222eee3e33211; no voters:
I20260619 20:38:51.568621 8131 raft_consensus.cc:2781] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:38:51.568670 8131 raft_consensus.cc:482] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:38:51.568698 8131 raft_consensus.cc:3037] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:38:51.568837 7800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e2fa55b13ba41e388dbd89fbce7ca8f" candidate_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611432b752984ba58352a16639bc031b" is_pre_election: true
I20260619 20:38:51.568984 7800 raft_consensus.cc:2454] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40af25e06e6f4c0ead5d34fe15bd172a in term 0.
I20260619 20:38:51.569615 8131 raft_consensus.cc:504] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:51.569765 8131 leader_election.cc:288] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [CANDIDATE]: Term 1 election: Requested vote from peers b4fc928f8db7494baeb222eee3e33211 (127.25.62.130:33293), 611432b752984ba58352a16639bc031b (127.25.62.129:43075)
I20260619 20:38:51.569958 7938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e2fa55b13ba41e388dbd89fbce7ca8f" candidate_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4fc928f8db7494baeb222eee3e33211"
I20260619 20:38:51.570026 7800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e2fa55b13ba41e388dbd89fbce7ca8f" candidate_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "611432b752984ba58352a16639bc031b"
I20260619 20:38:51.570057 7938 raft_consensus.cc:3037] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:38:51.570129 7800 raft_consensus.cc:3037] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:38:51.570770 7938 raft_consensus.cc:2454] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40af25e06e6f4c0ead5d34fe15bd172a in term 1.
I20260619 20:38:51.570815 7800 raft_consensus.cc:2454] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40af25e06e6f4c0ead5d34fe15bd172a in term 1.
I20260619 20:38:51.570956 8003 leader_election.cc:302] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40af25e06e6f4c0ead5d34fe15bd172a, b4fc928f8db7494baeb222eee3e33211; no voters:
I20260619 20:38:51.571043 8131 raft_consensus.cc:2781] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:38:51.571192 8131 raft_consensus.cc:686] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [term 1 LEADER]: Becoming Leader. State: Replica: 40af25e06e6f4c0ead5d34fe15bd172a, State: Running, Role: LEADER
I20260619 20:38:51.571297 8131 consensus_queue.cc:237] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:51.572072 7658 catalog_manager.cc:5495] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a reported cstate change: term changed from 0 to 1, leader changed from <none> to 40af25e06e6f4c0ead5d34fe15bd172a (127.25.62.131). New cstate: current_term: 1 leader_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } health_report { overall_health: UNKNOWN } } }
W20260619 20:38:51.600590 7847 tablet.cc:2369] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:38:51.605808 8117 tablet.cc:2369] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:38:51.607666 8123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.605341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34154 (local address 127.25.62.131:40199)
0619 20:38:51.605384 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:51.605386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:51.605400 (+ 14us) server_negotiation.cc:407] Connection header received
0619 20:38:51.605432 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:51.605433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:51.605449 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:51.605511 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:38:51.605613 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.606163 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.606889 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:51.607077 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:51.607149 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:51.607329 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:38:51.607423 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:38:51.607515 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:51.607543 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 20:38:51.607561 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:38:51.609292 7800 raft_consensus.cc:1261] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Refusing update from remote peer 40af25e06e6f4c0ead5d34fe15bd172a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:38:51.609333 7938 raft_consensus.cc:1261] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211 [term 1 FOLLOWER]: Refusing update from remote peer 40af25e06e6f4c0ead5d34fe15bd172a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:38:51.609627 8131 consensus_queue.cc:1035] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [LEADER]: Connected to new peer: Peer: permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:38:51.609779 8133 consensus_queue.cc:1035] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:38:51.611904 8137 mvcc.cc:205] Tried to move back new op lower bound from 7298668673468968960 to 7298668673316732928. Current Snapshot: MvccSnapshot[applied={T|T < 7298668673468968960}]
I20260619 20:38:51.612339 8138 mvcc.cc:205] Tried to move back new op lower bound from 7298668673468968960 to 7298668673316732928. Current Snapshot: MvccSnapshot[applied={T|T < 7298668673468968960}]
I20260619 20:38:51.613967 7658 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:60690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:38:51.614044 7658 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:60690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:38:51.614768 7658 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477: Sending DeleteTablet for 3 replicas of tablet 3e2fa55b13ba41e388dbd89fbce7ca8f
I20260619 20:38:51.615077 8050 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e2fa55b13ba41e388dbd89fbce7ca8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:38:51 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:34146
I20260619 20:38:51.615096 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 7696
I20260619 20:38:51.615630 7918 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e2fa55b13ba41e388dbd89fbce7ca8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:38:51 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:48374
I20260619 20:38:51.615846 8146 tablet_replica.cc:330] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211: stopping tablet replica
I20260619 20:38:51.615938 8146 raft_consensus.cc:2229] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:38:51.615954 8145 tablet_replica.cc:330] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a: stopping tablet replica
I20260619 20:38:51.616031 8146 raft_consensus.cc:2258] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:38:51.616043 8145 raft_consensus.cc:2229] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:38:51.616128 8145 raft_consensus.cc:2258] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:38:51.617105 8146 ts_tablet_manager.cc:1905] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:38:51.617223 8145 ts_tablet_manager.cc:1905] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:38:51.618863 8146 ts_tablet_manager.cc:1918] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:38:51.618937 8146 log.cc:1196] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/3e2fa55b13ba41e388dbd89fbce7ca8f
I20260619 20:38:51.619231 8146 ts_tablet_manager.cc:1939] T 3e2fa55b13ba41e388dbd89fbce7ca8f P b4fc928f8db7494baeb222eee3e33211: Deleting consensus metadata
I20260619 20:38:51.619268 8145 ts_tablet_manager.cc:1918] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:38:51.619344 8145 log.cc:1196] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/3e2fa55b13ba41e388dbd89fbce7ca8f
I20260619 20:38:51.619623 8145 ts_tablet_manager.cc:1939] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 40af25e06e6f4c0ead5d34fe15bd172a: Deleting consensus metadata
I20260619 20:38:51.619617 7642 catalog_manager.cc:4841] TS b4fc928f8db7494baeb222eee3e33211 (127.25.62.130:33293): tablet 3e2fa55b13ba41e388dbd89fbce7ca8f (table test-table [id=545e359c87ab4c81baa878f28c6b445a]) successfully deleted
I20260619 20:38:51.620003 7645 catalog_manager.cc:4841] TS 40af25e06e6f4c0ead5d34fe15bd172a (127.25.62.131:40199): tablet 3e2fa55b13ba41e388dbd89fbce7ca8f (table test-table [id=545e359c87ab4c81baa878f28c6b445a]) successfully deleted
W20260619 20:38:51.622326 7642 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.62.129:43075 (error 108)
I20260619 20:38:51.622642 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 7849
I20260619 20:38:51.622829 8124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.622685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:43075 (local address 127.0.0.1:53478)
0619 20:38:51.622753 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:51.622793 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.129:43075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 20:38:51.623262 7642 catalog_manager.cc:4584] TS 611432b752984ba58352a16639bc031b (127.25.62.129:43075): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e2fa55b13ba41e388dbd89fbce7ca8f: Network error: Client connection negotiation failed: client connection to 127.25.62.129:43075: connect: Connection refused (error 111)
I20260619 20:38:51.628109 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 7987
I20260619 20:38:51.632412 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 7625
I20260619 20:38:51.637413 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:46449
--webserver_interface=127.25.62.190
--webserver_port=41215
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:46449
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:38:51.716754 8148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:51.716949 8148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:51.716970 8148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:51.718124 8148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:38:51.718166 8148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:51.718180 8148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:51.718194 8148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:38:51.718205 8148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:38:51.719659 8148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:46449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:46449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=41215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:51.719903 8148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:51.720139 8148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:51.725072 8156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:51.725148 8154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:51.725203 8148 server_base.cc:1034] running on GCE node
W20260619 20:38:51.725067 8153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:51.725586 8148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:51.725828 8148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:51.726965 8148 hybrid_clock.cc:648] HybridClock initialized: now 1781901531726944 us; error 37 us; skew 500 ppm
Jun 19 20:38:51 dist-test-slave-0t40 krb5kdc[7080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901531, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:51.729494 8148 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:38:51.729960 8148 webserver.cc:460] Webserver started at http://127.25.62.190:41215/ using document root <none> and password file <none>
I20260619 20:38:51.730192 8148 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:51.730262 8148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:51.731580 8148 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:38:51.732226 8163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:51.732398 8148 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:51.732484 8148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "0bd0501973194ca1b6934e64e4e18477"
format_stamp: "Formatted at 2026-06-19 20:38:49 on dist-test-slave-0t40"
server_key: "66802977e17c225f20b886a8c457dabd"
server_key_iv: "d0320f6cd9752cb4f15e1ed4f1f0108d"
server_key_version: "encryptionkey@0"
I20260619 20:38:51.732817 8148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:51.753095 8148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:51.755522 8148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:51.755687 8148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:51.759756 8148 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:46449
I20260619 20:38:51.759806 8215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:46449 every 8 connection(s)
I20260619 20:38:51.760191 8148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:38:51.761871 8216 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:38:51.762030 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 8148
I20260619 20:38:51.762295 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:43075
--local_ip_for_outbound_sockets=127.25.62.129
--tserver_master_addrs=127.25.62.190:46449
--webserver_port=43001
--webserver_interface=127.25.62.129
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:38:51.762673 8216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477: Bootstrap starting.
I20260619 20:38:51.771584 8216 log.cc:824] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477: Log is configured to *not* fsync() on all Append() calls
I20260619 20:38:51.774358 8216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:38:51.774701 8216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477: Bootstrap complete.
I20260619 20:38:51.776985 8216 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd0501973194ca1b6934e64e4e18477" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46449 } }
I20260619 20:38:51.777314 8216 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd0501973194ca1b6934e64e4e18477, State: Initialized, Role: FOLLOWER
I20260619 20:38:51.777508 8216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd0501973194ca1b6934e64e4e18477" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46449 } }
I20260619 20:38:51.777597 8216 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:38:51.777663 8216 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:38:51.777716 8216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 2 FOLLOWER]: Advancing to term 3
I20260619 20:38:51.778780 8216 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd0501973194ca1b6934e64e4e18477" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46449 } }
I20260619 20:38:51.778872 8216 leader_election.cc:302] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bd0501973194ca1b6934e64e4e18477; no voters:
I20260619 20:38:51.779091 8216 leader_election.cc:288] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 20:38:51.779136 8220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 20:38:51.779263 8220 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [term 3 LEADER]: Becoming Leader. State: Replica: 0bd0501973194ca1b6934e64e4e18477, State: Running, Role: LEADER
I20260619 20:38:51.779353 8220 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd0501973194ca1b6934e64e4e18477" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46449 } }
I20260619 20:38:51.779386 8216 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:38:51.779784 8222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bd0501973194ca1b6934e64e4e18477. Latest consensus state: current_term: 3 leader_uuid: "0bd0501973194ca1b6934e64e4e18477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd0501973194ca1b6934e64e4e18477" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46449 } } }
I20260619 20:38:51.779870 8222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [sys.catalog]: This master's current role is: LEADER
I20260619 20:38:51.780143 8224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:38:51.780936 8224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:38:51.781082 8224 catalog_manager.cc:1259] Loaded cluster ID: 571a2301f0604d31afe00008dc4b5c88
I20260619 20:38:51.781121 8224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:38:51.781322 8221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0bd0501973194ca1b6934e64e4e18477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd0501973194ca1b6934e64e4e18477" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46449 } } }
F20260619 20:38:51.781359 8224 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
I20260619 20:38:51.781484 8221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd0501973194ca1b6934e64e4e18477 [sys.catalog]: This master's current role is: LEADER
@ 0x7fad5eaaf9ad google::LogMessage::Fail() at ??:0
@ 0x7fad5eab288a google::LogMessage::SendToLog() at ??:0
@ 0x7fad5eaaf4de google::LogMessage::Flush() at ??:0
@ 0x7fad5eab0a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fad669d40f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fad669dd8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fad6097d0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fad60972954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fad617d66db start_thread at ??:0
@ 0x7fad5d5ad61f clone at ??:0
W20260619 20:38:51.856720 8218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:51.856911 8218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:51.856948 8218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:51.858129 8218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:51.858171 8218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:51.858198 8218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:38:51.859629 8218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:43075
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=43001
--tserver_master_addrs=127.25.62.190:46449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:51.859861 8218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:51.860107 8218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:51.865204 8255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:51.865228 8252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:51.865195 8253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:51.865528 8218 server_base.cc:1034] running on GCE node
I20260619 20:38:51.865761 8218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:51.866035 8218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:51.867215 8218 hybrid_clock.cc:648] HybridClock initialized: now 1781901531867203 us; error 28 us; skew 500 ppm
Jun 19 20:38:51 dist-test-slave-0t40 krb5kdc[7080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901531, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:51.870285 8218 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:38:51.870847 8218 webserver.cc:460] Webserver started at http://127.25.62.129:43001/ using document root <none> and password file <none>
I20260619 20:38:51.871121 8218 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:51.871196 8218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:51.873028 8218 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:38:51.873879 8262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:51.874188 8218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:38:51.874289 8218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "611432b752984ba58352a16639bc031b"
format_stamp: "Formatted at 2026-06-19 20:38:50 on dist-test-slave-0t40"
server_key: "2e0797f7a91a232aed591f9d8cf2398e"
server_key_iv: "aa1f652056722269aa80a9a560849191"
server_key_version: "encryptionkey@0"
I20260619 20:38:51.874745 8218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:51.893333 8218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:51.895756 8218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:51.895918 8218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:51.896179 8218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:38:51.896710 8269 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 20:38:51.898252 8218 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 20:38:51.898317 8218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 20:38:51.898356 8218 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 20:38:51.899231 8218 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 20:38:51.899286 8218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 20:38:51.899348 8269 tablet_bootstrap.cc:492] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b: Bootstrap starting.
I20260619 20:38:51.907936 8218 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:43075
I20260619 20:38:51.908136 8376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:43075 every 8 connection(s)
I20260619 20:38:51.908420 8218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:38:51.909613 8269 log.cc:824] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:38:51.911667 8269 tablet_bootstrap.cc:492] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:38:51.911959 8269 tablet_bootstrap.cc:492] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b: Bootstrap complete.
I20260619 20:38:51.912906 8269 ts_tablet_manager.cc:1397] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.004s
I20260619 20:38:51.913900 8269 raft_consensus.cc:348] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:51.914115 8269 raft_consensus.cc:729] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 611432b752984ba58352a16639bc031b, State: Initialized, Role: FOLLOWER
I20260619 20:38:51.914245 8269 consensus_queue.cc:260] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:51.914525 8269 ts_tablet_manager.cc:1428] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b: Time spent starting tablet: real 0.002s user 0.004s sys 0.001s
I20260619 20:38:51.917585 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 8218
I20260619 20:38:51.917820 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:33293
--local_ip_for_outbound_sockets=127.25.62.130
--tserver_master_addrs=127.25.62.190:46449
--webserver_port=46525
--webserver_interface=127.25.62.130
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:38:51.976258 8379 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:38:51.909244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:35473)
0619 20:38:51.909470 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:51.909524 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:51.910142 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:51.976041 (+ 65899us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":109,"threads_started":1}
W20260619 20:38:51.976558 8377 heartbeater.cc:644] Failed to heartbeat to 127.25.62.190:46449 (0 consecutive failures): Network error: Failed to ping master at 127.25.62.190:46449: Client connection negotiation failed: client connection to 127.25.62.190:46449: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 20:38:51.976922 8379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.976804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:38675)
0619 20:38:51.976857 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:51.976900 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 20:38:51.977308 8379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:51.977174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:35847)
0619 20:38:51.977253 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:51.977284 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 20:38:51.977463 8377 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 20:38:51.998379 8382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:51.998611 8382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:51.998651 8382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:51.999794 8382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:51.999859 8382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:51.999907 8382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:38:52.001433 8382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:33293
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=46525
--tserver_master_addrs=127.25.62.190:46449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:52.001688 8382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:52.001909 8382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:38:52.007194 8390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:52.007229 8387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:52.007301 8388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:52.007485 8382 server_base.cc:1034] running on GCE node
I20260619 20:38:52.007691 8382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:52.007886 8382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:52.009022 8382 hybrid_clock.cc:648] HybridClock initialized: now 1781901532009013 us; error 27 us; skew 500 ppm
Jun 19 20:38:52 dist-test-slave-0t40 krb5kdc[7080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901532, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:52.011490 8382 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:38:52.011910 8382 webserver.cc:460] Webserver started at http://127.25.62.130:46525/ using document root <none> and password file <none>
I20260619 20:38:52.012127 8382 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:52.012195 8382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:52.013593 8382 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 20:38:52.014257 8397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:52.014474 8382 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:38:52.014595 8382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "b4fc928f8db7494baeb222eee3e33211"
format_stamp: "Formatted at 2026-06-19 20:38:50 on dist-test-slave-0t40"
server_key: "238acca44cff0c37f298d890327974ad"
server_key_iv: "52afb9e5d407145af999022c7829745a"
server_key_version: "encryptionkey@0"
I20260619 20:38:52.014942 8382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:52.031131 8382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:52.032998 8382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:52.033174 8382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:52.033438 8382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:38:52.033743 8382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:38:52.033799 8382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:52.033840 8382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:38:52.033869 8382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:52.040134 8382 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:33293
I20260619 20:38:52.040185 8510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:33293 every 8 connection(s)
I20260619 20:38:52.040567 8382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:38:52.041816 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:52.041246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:46721)
0619 20:38:52.041399 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:52.041474 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":75,"threads_started":1}
W20260619 20:38:52.042034 8511 heartbeater.cc:644] Failed to heartbeat to 127.25.62.190:46449 (0 consecutive failures): Network error: Failed to ping master at 127.25.62.190:46449: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
I20260619 20:38:52.042289 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:52.042206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:56545)
0619 20:38:52.042248 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:52.042272 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 20:38:52.042519 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:52.042467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:34851)
0619 20:38:52.042492 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:52.042509 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260619 20:38:52.042692 8511 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 20:38:52.042745 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 8382
I20260619 20:38:52.042970 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:40199
--local_ip_for_outbound_sockets=127.25.62.131
--tserver_master_addrs=127.25.62.190:46449
--webserver_port=44577
--webserver_interface=127.25.62.131
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:38:52.122097 8514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:38:52.122298 8514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:38:52.122320 8514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:38:52.123359 8514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:38:52.123389 8514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:38:52.123414 8514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:38:52.124727 8514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:40199
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=44577
--tserver_master_addrs=127.25.62.190:46449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:38:52.124895 8514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:38:52.125079 8514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:38:52.130183 8514 server_base.cc:1034] running on GCE node
W20260619 20:38:52.130179 8519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:52.130187 8522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:38:52.130375 8520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:38:52.130585 8514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:38:52.130828 8514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:38:52.131986 8514 hybrid_clock.cc:648] HybridClock initialized: now 1781901532131976 us; error 27 us; skew 500 ppm
Jun 19 20:38:52 dist-test-slave-0t40 krb5kdc[7080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901532, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:38:52.134369 8514 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:38:52.134826 8514 webserver.cc:460] Webserver started at http://127.25.62.131:44577/ using document root <none> and password file <none>
I20260619 20:38:52.135053 8514 fs_manager.cc:359] Metadata directory not provided
I20260619 20:38:52.135123 8514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:38:52.136464 8514 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:38:52.137161 8529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:52.137338 8514 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:38:52.137434 8514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "40af25e06e6f4c0ead5d34fe15bd172a"
format_stamp: "Formatted at 2026-06-19 20:38:50 on dist-test-slave-0t40"
server_key: "b2a1403bf6fd81b55a77eeda68c9c176"
server_key_iv: "6ac2a95b1f4515bb18f89b129bba9efb"
server_key_version: "encryptionkey@0"
I20260619 20:38:52.137760 8514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:38:52.149397 8514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:38:52.151418 8514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:38:52.151579 8514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:38:52.151808 8514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:38:52.152117 8514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:38:52.152176 8514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:52.152216 8514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:38:52.152243 8514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:38:52.158114 8514 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:40199
I20260619 20:38:52.158161 8642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:40199 every 8 connection(s)
I20260619 20:38:52.158484 8514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:38:52.159505 8645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:52.159123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:52679)
0619 20:38:52.159279 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:52.159373 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
W20260619 20:38:52.159850 8643 heartbeater.cc:644] Failed to heartbeat to 127.25.62.190:46449 (0 consecutive failures): Network error: Failed to ping master at 127.25.62.190:46449: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
I20260619 20:38:52.160113 8645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:52.160021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:51303)
0619 20:38:52.160069 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:52.160097 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 20:38:52.160393 8645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:52.160306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:57635)
0619 20:38:52.160349 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:52.160370 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 20:38:52.160488 8643 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 20:38:52.167622 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 8514
I20260619 20:38:52.978080 8647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:52.977832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:40627)
0619 20:38:52.977984 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:52.978032 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":66,"threads_started":1}
I20260619 20:38:53.043205 8648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:53.043005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:49255)
0619 20:38:53.043123 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:53.043158 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":52,"threads_started":1}
I20260619 20:38:53.160943 8649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:53.160769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:49353)
0619 20:38:53.160868 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:53.160897 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":58,"threads_started":1}
I20260619 20:38:53.373255 8650 raft_consensus.cc:482] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:38:53.373379 8650 raft_consensus.cc:504] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:53.373744 8650 leader_election.cc:288] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4fc928f8db7494baeb222eee3e33211 (127.25.62.130:33293), 40af25e06e6f4c0ead5d34fe15bd172a (127.25.62.131:40199)
Jun 19 20:38:53 dist-test-slave-0t40 krb5kdc[7080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901531, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.131@KRBTEST.COM
Jun 19 20:38:53 dist-test-slave-0t40 krb5kdc[7080](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901531, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.130@KRBTEST.COM
I20260619 20:38:53.384187 8653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:53.374087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:42733 (local address 127.25.62.130:33293)
0619 20:38:53.374234 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:53.374242 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:53.374273 (+ 31us) server_negotiation.cc:407] Connection header received
0619 20:38:53.374688 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:53.374690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:53.374724 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:53.374870 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:53.375571 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:53.376397 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:53.377460 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:53.377918 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:53.378161 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:53.381031 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:53.381054 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:53.381054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:53.381086 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:53.382717 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:53.383095 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:53.383097 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:53.383097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:53.383218 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:53.383438 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:53.383439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:53.383439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:53.383666 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:53.383743 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:53.383959 (+ 216us) server_negotiation.cc:299] Negotiation successful
0619 20:38:53.384076 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260619 20:38:53.384176 8647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:53.373915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:40199 (local address 127.25.62.129:41745)
0619 20:38:53.373975 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:53.373988 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:53.374057 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:53.374904 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:53.374906 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:53.375271 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:53.375443 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:53.375449 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:53.376507 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:53.376508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:53.377354 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:53.377356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:53.378010 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:53.378011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:53.378124 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:53.378717 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:53.378721 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:53.380537 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:53.382571 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:53.382578 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:53.382580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:53.382894 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:53.383146 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:53.383147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:53.383148 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:53.383266 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:53.383658 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:53.383664 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:53.383801 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 20:38:53.383904 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 20:38:53.384081 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":4}
I20260619 20:38:53.384208 8652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:53.374042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:41745 (local address 127.25.62.131:40199)
0619 20:38:53.374179 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 20:38:53.374192 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 20:38:53.374227 (+ 35us) server_negotiation.cc:407] Connection header received
0619 20:38:53.374672 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:38:53.374674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:38:53.374715 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:38:53.374860 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:38:53.375566 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:53.376399 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:53.377438 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:38:53.377907 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:38:53.378045 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:53.380633 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:38:53.380636 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:38:53.380637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:38:53.380646 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:38:53.382492 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:53.382968 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:53.382972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:53.382972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:53.383080 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:38:53.383338 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:38:53.383339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:38:53.383340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:38:53.383589 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:38:53.383647 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:38:53.383958 (+ 311us) server_negotiation.cc:299] Negotiation successful
0619 20:38:53.384077 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260619 20:38:53.384287 8651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:53.373935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:33293 (local address 127.25.62.129:42733)
0619 20:38:53.374026 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:53.374038 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:38:53.374088 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:38:53.374905 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:38:53.374907 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:38:53.375274 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:38:53.375443 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:53.375449 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:53.376498 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:53.376499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:53.377355 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:38:53.377358 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:38:53.378138 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:38:53.378139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:38:53.378184 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:38:53.378688 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:38:53.378698 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:38:53.380924 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:38:53.382797 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:53.382798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:53.382798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:53.383019 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:53.383288 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:38:53.383289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:38:53.383289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:38:53.383374 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:38:53.383744 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:38:53.383744 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:38:53.383795 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 20:38:53.383904 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 20:38:53.384158 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":185,"thread_start_us":43,"threads_started":1}
W20260619 20:38:53.384900 8266 leader_election.cc:341] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40af25e06e6f4c0ead5d34fe15bd172a (127.25.62.131:40199): Not found: Tablet not found: 3e2fa55b13ba41e388dbd89fbce7ca8f
W20260619 20:38:53.385473 8263 leader_election.cc:341] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b4fc928f8db7494baeb222eee3e33211 (127.25.62.130:33293): Not found: Tablet not found: 3e2fa55b13ba41e388dbd89fbce7ca8f
I20260619 20:38:53.385531 8263 leader_election.cc:302] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611432b752984ba58352a16639bc031b; no voters: 40af25e06e6f4c0ead5d34fe15bd172a, b4fc928f8db7494baeb222eee3e33211
I20260619 20:38:53.385635 8650 raft_consensus.cc:2726] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:38:53.978770 8654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:53.978565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:46023)
0619 20:38:53.978682 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:53.978726 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
I20260619 20:38:54.043788 8655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:54.043604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:48509)
0619 20:38:54.043714 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:54.043752 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
I20260619 20:38:54.161598 8656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:54.161388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:33321)
0619 20:38:54.161522 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:54.161564 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
I20260619 20:38:54.979562 8657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:54.979345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:52187)
0619 20:38:54.979479 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:54.979516 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260619 20:38:55.044414 8658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:55.044216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:59501)
0619 20:38:55.044350 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:55.044379 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260619 20:38:55.162228 8659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:55.162002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:36679)
0619 20:38:55.162141 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:55.162182 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":60,"threads_started":1}
I20260619 20:38:55.564169 8660 raft_consensus.cc:482] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:38:55.564268 8660 raft_consensus.cc:504] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:55.564401 8660 leader_election.cc:288] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4fc928f8db7494baeb222eee3e33211 (127.25.62.130:33293), 40af25e06e6f4c0ead5d34fe15bd172a (127.25.62.131:40199)
W20260619 20:38:55.564812 8263 leader_election.cc:341] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b4fc928f8db7494baeb222eee3e33211 (127.25.62.130:33293): Not found: Tablet not found: 3e2fa55b13ba41e388dbd89fbce7ca8f
W20260619 20:38:55.564893 8266 leader_election.cc:341] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40af25e06e6f4c0ead5d34fe15bd172a (127.25.62.131:40199): Not found: Tablet not found: 3e2fa55b13ba41e388dbd89fbce7ca8f
I20260619 20:38:55.564971 8266 leader_election.cc:302] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611432b752984ba58352a16639bc031b; no voters: 40af25e06e6f4c0ead5d34fe15bd172a, b4fc928f8db7494baeb222eee3e33211
I20260619 20:38:55.565070 8660 raft_consensus.cc:2726] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:38:55.980243 8661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:55.980003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:50837)
0619 20:38:55.980138 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:55.980177 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":62,"threads_started":1}
I20260619 20:38:56.045009 8662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:56.044827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:39695)
0619 20:38:56.044938 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:56.044975 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":52,"threads_started":1}
I20260619 20:38:56.162925 8663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:56.162727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:37807)
0619 20:38:56.162839 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:56.162874 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":66,"threads_started":1}
I20260619 20:38:56.981026 8664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:56.980814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:40643)
0619 20:38:56.980948 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:56.980979 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260619 20:38:57.045639 8665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:57.045448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:37445)
0619 20:38:57.045567 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:57.045602 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":56,"threads_started":1}
I20260619 20:38:57.163594 8666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:57.163371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:34869)
0619 20:38:57.163508 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:57.163550 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260619 20:38:57.173519 8667 raft_consensus.cc:482] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:38:57.173607 8667 raft_consensus.cc:504] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:38:57.173760 8667 leader_election.cc:288] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4fc928f8db7494baeb222eee3e33211 (127.25.62.130:33293), 40af25e06e6f4c0ead5d34fe15bd172a (127.25.62.131:40199)
W20260619 20:38:57.174139 8266 leader_election.cc:341] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40af25e06e6f4c0ead5d34fe15bd172a (127.25.62.131:40199): Not found: Tablet not found: 3e2fa55b13ba41e388dbd89fbce7ca8f
W20260619 20:38:57.174249 8263 leader_election.cc:341] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b4fc928f8db7494baeb222eee3e33211 (127.25.62.130:33293): Not found: Tablet not found: 3e2fa55b13ba41e388dbd89fbce7ca8f
I20260619 20:38:57.174327 8263 leader_election.cc:302] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611432b752984ba58352a16639bc031b; no voters: 40af25e06e6f4c0ead5d34fe15bd172a, b4fc928f8db7494baeb222eee3e33211
I20260619 20:38:57.174414 8667 raft_consensus.cc:2726] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:38:57.981750 8668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:57.981545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:59817)
0619 20:38:57.981664 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:57.981705 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
I20260619 20:38:58.046290 8669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:58.046079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:37767)
0619 20:38:58.046205 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:58.046252 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":57,"threads_started":1}
I20260619 20:38:58.164305 8670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:58.164076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:36431)
0619 20:38:58.164210 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:58.164258 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":55,"threads_started":1}
I20260619 20:38:58.982450 8671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:58.982226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:45401)
0619 20:38:58.982368 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:58.982405 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":62,"threads_started":1}
I20260619 20:38:59.046905 8672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:59.046712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:53929)
0619 20:38:59.046838 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:59.046866 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":59,"threads_started":1}
I20260619 20:38:59.164987 8673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:59.164766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:35833)
0619 20:38:59.164898 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:59.164945 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":58,"threads_started":1}
I20260619 20:38:59.983269 8674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:38:59.983025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:52027)
0619 20:38:59.983183 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 20:38:59.983223 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":76,"threads_started":1}
I20260619 20:39:00.047591 8675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:00.047365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:53483)
0619 20:39:00.047501 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:00.047538 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":58,"threads_started":1}
I20260619 20:39:00.165807 8676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:00.165583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:44077)
0619 20:39:00.165713 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:00.165762 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":56,"threads_started":1}
I20260619 20:39:00.983981 8677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:00.983755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:56483)
0619 20:39:00.983887 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:00.983927 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":62,"threads_started":1}
I20260619 20:39:01.048336 8678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:01.048113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:47455)
0619 20:39:01.048234 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:01.048272 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
I20260619 20:39:01.166520 8679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:01.166335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:43107)
0619 20:39:01.166451 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:01.166480 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
I20260619 20:39:01.567059 8680 raft_consensus.cc:482] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:39:01.567165 8680 raft_consensus.cc:504] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:39:01.567425 8680 leader_election.cc:288] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4fc928f8db7494baeb222eee3e33211 (127.25.62.130:33293), 40af25e06e6f4c0ead5d34fe15bd172a (127.25.62.131:40199)
W20260619 20:39:01.567812 8266 leader_election.cc:341] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40af25e06e6f4c0ead5d34fe15bd172a (127.25.62.131:40199): Not found: Tablet not found: 3e2fa55b13ba41e388dbd89fbce7ca8f
W20260619 20:39:01.567886 8263 leader_election.cc:341] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b4fc928f8db7494baeb222eee3e33211 (127.25.62.130:33293): Not found: Tablet not found: 3e2fa55b13ba41e388dbd89fbce7ca8f
I20260619 20:39:01.567962 8263 leader_election.cc:302] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611432b752984ba58352a16639bc031b; no voters: 40af25e06e6f4c0ead5d34fe15bd172a, b4fc928f8db7494baeb222eee3e33211
I20260619 20:39:01.568053 8680 raft_consensus.cc:2726] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:39:01.984679 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:01.984451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:59841)
0619 20:39:01.984602 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:01.984634 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260619 20:39:02.049096 8682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:02.048836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:49007)
0619 20:39:02.048961 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:02.048994 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260619 20:39:02.167210 8683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:02.166945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:60923)
0619 20:39:02.167128 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:02.167167 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":58,"threads_started":1}
I20260619 20:39:02.985397 8684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:02.985197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:44691)
0619 20:39:02.985316 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:02.985352 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260619 20:39:03.049702 8685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:03.049520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:37109)
0619 20:39:03.049630 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:03.049663 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
I20260619 20:39:03.167820 8686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:03.167620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:54989)
0619 20:39:03.167737 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:03.167773 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260619 20:39:03.986019 8687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:03.985810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:35781)
0619 20:39:03.985935 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:03.985974 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260619 20:39:04.050495 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:04.050239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:57123)
0619 20:39:04.050401 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:04.050447 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":72,"threads_started":1}
I20260619 20:39:04.168458 8689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:04.168268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:51235)
0619 20:39:04.168385 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:04.168416 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
I20260619 20:39:04.830322 8690 raft_consensus.cc:482] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:39:04.830426 8690 raft_consensus.cc:504] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4fc928f8db7494baeb222eee3e33211" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33293 } } peers { permanent_uuid: "40af25e06e6f4c0ead5d34fe15bd172a" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40199 } } peers { permanent_uuid: "611432b752984ba58352a16639bc031b" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 43075 } }
I20260619 20:39:04.830636 8690 leader_election.cc:288] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b4fc928f8db7494baeb222eee3e33211 (127.25.62.130:33293), 40af25e06e6f4c0ead5d34fe15bd172a (127.25.62.131:40199)
W20260619 20:39:04.831096 8266 leader_election.cc:341] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40af25e06e6f4c0ead5d34fe15bd172a (127.25.62.131:40199): Not found: Tablet not found: 3e2fa55b13ba41e388dbd89fbce7ca8f
W20260619 20:39:04.831213 8263 leader_election.cc:341] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b4fc928f8db7494baeb222eee3e33211 (127.25.62.130:33293): Not found: Tablet not found: 3e2fa55b13ba41e388dbd89fbce7ca8f
I20260619 20:39:04.831272 8263 leader_election.cc:302] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 611432b752984ba58352a16639bc031b; no voters: 40af25e06e6f4c0ead5d34fe15bd172a, b4fc928f8db7494baeb222eee3e33211
I20260619 20:39:04.831367 8690 raft_consensus.cc:2726] T 3e2fa55b13ba41e388dbd89fbce7ca8f P 611432b752984ba58352a16639bc031b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:39:04.986864 8691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:04.986628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:49249)
0619 20:39:04.986753 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:04.986817 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":59,"threads_started":1}
I20260619 20:39:05.051339 8692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:05.051088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:59307)
0619 20:39:05.051253 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:05.051296 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":89,"threads_started":1}
I20260619 20:39:05.169387 8693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:05.169102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:50797)
0619 20:39:05.169280 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:05.169334 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":59,"threads_started":1}
I20260619 20:39:05.987659 8694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:05.987399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:59639)
0619 20:39:05.987555 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:05.987609 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":61,"threads_started":1}
I20260619 20:39:06.052049 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:06.051822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:40775)
0619 20:39:06.051966 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:06.052009 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":70,"threads_started":1}
I20260619 20:39:06.170182 8696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:06.169955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:45205)
0619 20:39:06.170093 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:06.170140 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":57,"threads_started":1}
I20260619 20:39:06.988478 8697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:06.988237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.129:50147)
0619 20:39:06.988392 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:06.988434 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":62,"threads_started":1}
I20260619 20:39:07.052858 8698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:07.052621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.130:42089)
0619 20:39:07.052769 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:07.052812 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":61,"threads_started":1}
I20260619 20:39:07.170955 8699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:07.170732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46449 (local address 127.25.62.131:51623)
0619 20:39:07.170871 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:07.170911 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
I20260619 20:39:07.172231 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 8218
I20260619 20:39:07.177846 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 8382
I20260619 20:39:07.182194 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 8514
2026-06-19T20:39:07Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18429 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:39:07 dist-test-slave-0t40 krb5kdc[8702](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:39:07 dist-test-slave-0t40 krb5kdc[8702](info): set up 2 sockets
Jun 19 20:39:07 dist-test-slave-0t40 krb5kdc[8702](info): commencing operation
krb5kdc: starting...
W20260619 20:39:08.237447 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:39:08 dist-test-slave-0t40 krb5kdc[8702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:39:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:39:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:39:08.355579 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:46749
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:46749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:39:08.434796 8715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:39:08.434995 8715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:39:08.435017 8715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:39:08.436142 8715 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 20:39:08.436182 8715 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 20:39:08.436194 8715 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 20:39:08.436229 8715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:39:08.436260 8715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:39:08.436280 8715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:39:08.436305 8715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:39:08.436324 8715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:39:08.437758 8715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:46749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:46749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:39:08.437999 8715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:39:08.438235 8715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:39:08.443432 8723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:39:08.443445 8721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:39:08.443466 8715 server_base.cc:1034] running on GCE node
W20260619 20:39:08.443415 8720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:39:08.443894 8715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:39:08.444196 8715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:39:08.445358 8715 hybrid_clock.cc:648] HybridClock initialized: now 1781901548445334 us; error 42 us; skew 500 ppm
Jun 19 20:39:08 dist-test-slave-0t40 krb5kdc[8702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901548, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:39:08.447803 8715 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:39:08.448238 8715 webserver.cc:460] Webserver started at http://127.25.62.190:39817/ using document root <none> and password file <none>
I20260619 20:39:08.448472 8715 fs_manager.cc:359] Metadata directory not provided
I20260619 20:39:08.448539 8715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:39:08.448693 8715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:39:08.449877 8715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "9f6af0c094d2409e8a9f27895b305501"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "4231f42bd033cbe7a27d55b5f3d64fc7"
server_key_iv: "c9f269a8104bb4f75c68e0d2ce48af97"
server_key_version: "encryptionkey@0"
I20260619 20:39:08.450232 8715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "9f6af0c094d2409e8a9f27895b305501"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "4231f42bd033cbe7a27d55b5f3d64fc7"
server_key_iv: "c9f269a8104bb4f75c68e0d2ce48af97"
server_key_version: "encryptionkey@0"
I20260619 20:39:08.451560 8715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:39:08.452360 8730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:08.452546 8715 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:39:08.452651 8715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "9f6af0c094d2409e8a9f27895b305501"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "4231f42bd033cbe7a27d55b5f3d64fc7"
server_key_iv: "c9f269a8104bb4f75c68e0d2ce48af97"
server_key_version: "encryptionkey@0"
I20260619 20:39:08.452723 8715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:39:08.475519 8715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:39:08.477512 8715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:39:08.477669 8715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:39:08.481541 8715 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:46749
I20260619 20:39:08.481590 8782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:46749 every 8 connection(s)
I20260619 20:39:08.481950 8715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:39:08.482798 8783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:39:08.485144 8783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501: Bootstrap starting.
I20260619 20:39:08.485723 8783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501: Neither blocks nor log segments found. Creating new log.
I20260619 20:39:08.486035 8783 log.cc:824] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501: Log is configured to *not* fsync() on all Append() calls
I20260619 20:39:08.486701 8783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501: No bootstrap required, opened a new log
I20260619 20:39:08.488284 8783 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6af0c094d2409e8a9f27895b305501" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46749 } }
I20260619 20:39:08.488420 8783 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:39:08.488443 8783 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f6af0c094d2409e8a9f27895b305501, State: Initialized, Role: FOLLOWER
I20260619 20:39:08.488610 8783 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6af0c094d2409e8a9f27895b305501" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46749 } }
I20260619 20:39:08.488672 8783 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:39:08.488691 8783 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:39:08.488770 8783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:39:08.489300 8783 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6af0c094d2409e8a9f27895b305501" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46749 } }
I20260619 20:39:08.489406 8783 leader_election.cc:302] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f6af0c094d2409e8a9f27895b305501; no voters:
I20260619 20:39:08.489595 8783 leader_election.cc:288] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:39:08.489635 8786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:39:08.489817 8786 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 1 LEADER]: Becoming Leader. State: Replica: 9f6af0c094d2409e8a9f27895b305501, State: Running, Role: LEADER
I20260619 20:39:08.489867 8783 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:39:08.490139 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 8715
I20260619 20:39:08.490240 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:39:08.490010 8786 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6af0c094d2409e8a9f27895b305501" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46749 } }
I20260619 20:39:08.490475 25850 external_mini_cluster.cc:1428] Setting key 681bde01fa19e1cd88577f9fd9fc65ed
I20260619 20:39:08.490707 8787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f6af0c094d2409e8a9f27895b305501" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6af0c094d2409e8a9f27895b305501" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46749 } } }
I20260619 20:39:08.490876 8787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [sys.catalog]: This master's current role is: LEADER
I20260619 20:39:08.490913 8788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f6af0c094d2409e8a9f27895b305501. Latest consensus state: current_term: 1 leader_uuid: "9f6af0c094d2409e8a9f27895b305501" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6af0c094d2409e8a9f27895b305501" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46749 } } }
I20260619 20:39:08.491012 8788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [sys.catalog]: This master's current role is: LEADER
I20260619 20:39:08.491496 8797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:39:08.492030 8797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:39:08.495901 8797 catalog_manager.cc:1347] Generated new cluster ID: c3518f7e531849e7a5f827d6bc0af70d
I20260619 20:39:08.495976 8797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:39:08 dist-test-slave-0t40 krb5kdc[8702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:39:08.502156 8800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:08.491096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41356 (local address 127.25.62.190:46749)
0619 20:39:08.491259 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:08.491283 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:08.491315 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:39:08.491536 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:08.491543 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:08.491604 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:08.491736 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:39:08.492504 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:08.493536 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:08.494617 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:08.495000 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:08.495071 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:08.497962 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:39:08.497966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:39:08.497966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:39:08.497978 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:39:08.500164 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:08.500568 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:08.500571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:08.500572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:08.500683 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:08.500906 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:08.500907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:08.500907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:08.501229 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:39:08.501372 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:08.501819 (+ 447us) server_negotiation.cc:299] Negotiation successful
0619 20:39:08.501957 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":66,"threads_started":1}
I20260619 20:39:08.507124 8797 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:39:08.507822 8797 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:39:08.516348 8797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501: Generated new TSK 0
I20260619 20:39:08.516551 8797 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
I20260619 20:39:08.545344 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:46749
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:39:08.625756 8809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:39:08.625994 8809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:39:08.626032 8809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:39:08.627166 8809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:39:08.627233 8809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:39:08.627279 8809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:39:08.628721 8809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:46749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:39:08.628985 8809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:39:08.629225 8809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:39:08.634042 8814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:39:08.634063 8815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:39:08.634042 8817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:39:08.634521 8809 server_base.cc:1034] running on GCE node
I20260619 20:39:08.634680 8809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:39:08.634862 8809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:39:08.635990 8809 hybrid_clock.cc:648] HybridClock initialized: now 1781901548635982 us; error 26 us; skew 500 ppm
Jun 19 20:39:08 dist-test-slave-0t40 krb5kdc[8702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901548, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:39:08.638439 8809 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:39:08.638851 8809 webserver.cc:460] Webserver started at http://127.25.62.129:37511/ using document root <none> and password file <none>
I20260619 20:39:08.639083 8809 fs_manager.cc:359] Metadata directory not provided
I20260619 20:39:08.639129 8809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:39:08.639257 8809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:39:08.640293 8809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "eb7e685f461044bb9c20a8a8838efe61"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "27cd64a84fac91b22f72739059f2c960"
server_key_iv: "584a70992d78548f10a5524a0c7203b7"
server_key_version: "encryptionkey@0"
I20260619 20:39:08.640646 8809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "eb7e685f461044bb9c20a8a8838efe61"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "27cd64a84fac91b22f72739059f2c960"
server_key_iv: "584a70992d78548f10a5524a0c7203b7"
server_key_version: "encryptionkey@0"
I20260619 20:39:08.642009 8809 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 20:39:08.642776 8824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:08.643002 8809 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:39:08.643069 8809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "eb7e685f461044bb9c20a8a8838efe61"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "27cd64a84fac91b22f72739059f2c960"
server_key_iv: "584a70992d78548f10a5524a0c7203b7"
server_key_version: "encryptionkey@0"
I20260619 20:39:08.643147 8809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:39:08.680755 8809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:39:08.682706 8809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:39:08.682861 8809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:39:08.683097 8809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:39:08.683416 8809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:39:08.683471 8809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:08.683514 8809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:39:08.683544 8809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:08.689219 8809 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:39417
I20260619 20:39:08.689273 8937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:39417 every 8 connection(s)
I20260619 20:39:08.689577 8809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:39:08.690693 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 8809
I20260619 20:39:08.690796 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:39:08.691009 25850 external_mini_cluster.cc:1428] Setting key 0de74e826586bb98055859ba73d8e34a
Jun 19 20:39:08 dist-test-slave-0t40 krb5kdc[8702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901548, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.62.130@KRBTEST.COM; defaulting to no policy
I20260619 20:39:08.699740 8800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:08.690259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:55445 (local address 127.25.62.190:46749)
0619 20:39:08.690309 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:08.690312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:08.690560 (+ 248us) server_negotiation.cc:407] Connection header received
0619 20:39:08.691382 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:08.691384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:08.691399 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:08.691473 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:39:08.692738 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:08.693417 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:08.694416 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:08.694634 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:08.694721 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:08.696877 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:39:08.696878 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:39:08.696878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:39:08.696890 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:39:08.698337 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:08.698754 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:08.698756 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:08.698756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:08.698816 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:08.699088 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:08.699089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:08.699089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:08.699200 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:39:08.699291 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:08.699564 (+ 273us) server_negotiation.cc:299] Negotiation successful
0619 20:39:08.699648 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:39:08.699972 8940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:08.690246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:55445)
0619 20:39:08.690406 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:08.690441 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:08.691270 (+ 829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:08.691544 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:08.691548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:08.692371 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:39:08.692647 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:08.692653 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:08.693519 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:08.693520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:08.694309 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:08.694312 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:08.694733 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:08.694734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:08.694875 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:08.695380 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:39:08.695388 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:39:08.696781 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:39:08.698439 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:39:08.698445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:39:08.698448 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:39:08.698663 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:39:08.698909 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:39:08.698910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:39:08.698910 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:39:08.699013 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:39:08.699287 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:39:08.699294 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:39:08.699432 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 20:39:08.699543 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 20:39:08.699698 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260619 20:39:08.700472 8938 heartbeater.cc:347] Connected to a master server at 127.25.62.190:46749
I20260619 20:39:08.700552 8938 heartbeater.cc:464] Registering TS with master...
I20260619 20:39:08.700709 8938 heartbeater.cc:511] Master 127.25.62.190:46749 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.130@KRBTEST.COM" created.
I20260619 20:39:08.701398 8747 ts_manager.cc:194] Registered new tserver with Master: eb7e685f461044bb9c20a8a8838efe61 (127.25.62.129:39417)
I20260619 20:39:08.702090 8747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:55445
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
I20260619 20:39:08.716882 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:46749
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:39:08.797482 8943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:39:08.797701 8943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:39:08.797741 8943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:39:08.798889 8943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:39:08.798954 8943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:39:08.798997 8943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:39:08.800436 8943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:46749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:39:08.800700 8943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:39:08.800921 8943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:39:08.805871 8951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:39:08.805848 8949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:39:08.805848 8948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:39:08.806056 8943 server_base.cc:1034] running on GCE node
I20260619 20:39:08.806298 8943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:39:08.806491 8943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:39:08.807662 8943 hybrid_clock.cc:648] HybridClock initialized: now 1781901548807642 us; error 40 us; skew 500 ppm
Jun 19 20:39:08 dist-test-slave-0t40 krb5kdc[8702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901548, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:39:08.810233 8943 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:39:08.810672 8943 webserver.cc:460] Webserver started at http://127.25.62.130:33683/ using document root <none> and password file <none>
I20260619 20:39:08.810909 8943 fs_manager.cc:359] Metadata directory not provided
I20260619 20:39:08.810954 8943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:39:08.811082 8943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:39:08.812114 8943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "14d00dba0a6c4f8e9470116cefa36c69"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "0f7dde673f0c82b0b72f05ce7dbd33b6"
server_key_iv: "6060b90292c99ab92382dbac44e2c5f6"
server_key_version: "encryptionkey@0"
I20260619 20:39:08.812458 8943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "14d00dba0a6c4f8e9470116cefa36c69"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "0f7dde673f0c82b0b72f05ce7dbd33b6"
server_key_iv: "6060b90292c99ab92382dbac44e2c5f6"
server_key_version: "encryptionkey@0"
I20260619 20:39:08.813937 8943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:39:08.814653 8958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:08.814829 8943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:08.814929 8943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "14d00dba0a6c4f8e9470116cefa36c69"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "0f7dde673f0c82b0b72f05ce7dbd33b6"
server_key_iv: "6060b90292c99ab92382dbac44e2c5f6"
server_key_version: "encryptionkey@0"
I20260619 20:39:08.815007 8943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:39:08.829567 8943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:39:08.831511 8943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:39:08.831615 8943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:39:08.831840 8943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:39:08.832146 8943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:39:08.832206 8943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:08.832254 8943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:39:08.832281 8943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:08.838207 8943 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:44777
I20260619 20:39:08.838268 9071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:44777 every 8 connection(s)
I20260619 20:39:08.838598 8943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:39:08.841970 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 8943
I20260619 20:39:08.842077 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:39:08.842288 25850 external_mini_cluster.cc:1428] Setting key 2557f44d1526a89a9d052fe45797199c
Jun 19 20:39:08 dist-test-slave-0t40 krb5kdc[8702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901548, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:39:08.848620 8800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:08.839346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:50969 (local address 127.25.62.190:46749)
0619 20:39:08.839435 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:08.839437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:08.839620 (+ 183us) server_negotiation.cc:407] Connection header received
0619 20:39:08.840156 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:08.840158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:08.840172 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:08.840244 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:39:08.841200 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:08.841719 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:08.842902 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:08.843107 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:08.843202 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:08.845712 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:39:08.845714 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:39:08.845714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:39:08.845722 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:39:08.847255 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:08.847672 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:08.847673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:08.847674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:08.847731 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:08.847986 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:08.847987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:08.847988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:08.848103 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:39:08.848178 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:08.848445 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 20:39:08.848531 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 20:39:08.848827 9074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:08.839261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:50969)
0619 20:39:08.839472 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:08.839514 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:08.840043 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:08.840305 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:08.840314 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:08.840886 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:39:08.841089 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:08.841094 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:08.841837 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:08.841838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:08.842801 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:08.842804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:08.843215 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:08.843216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:08.843364 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:08.843911 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:39:08.843919 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:39:08.845608 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:39:08.847350 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:39:08.847357 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:39:08.847359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:39:08.847583 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:39:08.847804 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:39:08.847805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:39:08.847806 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:39:08.847919 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:39:08.848170 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:39:08.848176 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:39:08.848317 (+ 141us) client_negotiation.cc:769] Sending connection context
0619 20:39:08.848426 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 20:39:08.848582 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":90,"threads_started":1}
I20260619 20:39:08.849287 9072 heartbeater.cc:347] Connected to a master server at 127.25.62.190:46749
I20260619 20:39:08.849406 9072 heartbeater.cc:464] Registering TS with master...
I20260619 20:39:08.849606 9072 heartbeater.cc:511] Master 127.25.62.190:46749 requested a full tablet report, sending...
I20260619 20:39:08.849990 8747 ts_manager.cc:194] Registered new tserver with Master: 14d00dba0a6c4f8e9470116cefa36c69 (127.25.62.130:44777)
I20260619 20:39:08.850365 8747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:50969
WARNING: no policy specified for kudu/127.25.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
I20260619 20:39:08.868204 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:46749
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:39:08.949545 9077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:39:08.949728 9077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:39:08.949749 9077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:39:08.950867 9077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:39:08.950908 9077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:39:08.950959 9077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:39:08.952443 9077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:46749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:39:08.952677 9077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:39:08.952903 9077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:39:08.957862 9085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:39:08.957873 9083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:39:08.957890 9077 server_base.cc:1034] running on GCE node
W20260619 20:39:08.957888 9082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:39:08.958302 9077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:39:08.958503 9077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:39:08.959637 9077 hybrid_clock.cc:648] HybridClock initialized: now 1781901548959630 us; error 26 us; skew 500 ppm
Jun 19 20:39:08 dist-test-slave-0t40 krb5kdc[8702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901548, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:39:08.962322 9077 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:39:08.962806 9077 webserver.cc:460] Webserver started at http://127.25.62.131:44565/ using document root <none> and password file <none>
I20260619 20:39:08.963049 9077 fs_manager.cc:359] Metadata directory not provided
I20260619 20:39:08.963095 9077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:39:08.963219 9077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:39:08.964221 9077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "033c91b091ed4080b835cb06c5c976c5"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "b93e39b2351f193b70454e6aec133fc0"
server_key_iv: "4e2924017a5500ffdfd132f0b04902ba"
server_key_version: "encryptionkey@0"
I20260619 20:39:08.964562 9077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "033c91b091ed4080b835cb06c5c976c5"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "b93e39b2351f193b70454e6aec133fc0"
server_key_iv: "4e2924017a5500ffdfd132f0b04902ba"
server_key_version: "encryptionkey@0"
I20260619 20:39:08.965947 9077 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:39:08.966727 9092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:08.966907 9077 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:39:08.967003 9077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "033c91b091ed4080b835cb06c5c976c5"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "b93e39b2351f193b70454e6aec133fc0"
server_key_iv: "4e2924017a5500ffdfd132f0b04902ba"
server_key_version: "encryptionkey@0"
I20260619 20:39:08.967075 9077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:39:09.002723 9077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:39:09.005213 9077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:39:09.005373 9077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:39:09.005609 9077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:39:09.005914 9077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:39:09.005946 9077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:09.006006 9077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:39:09.006032 9077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:09.012039 9077 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:40907
I20260619 20:39:09.012081 9205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:40907 every 8 connection(s)
I20260619 20:39:09.012424 9077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:39:09.013144 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 9077
I20260619 20:39:09.013233 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:39:09.013437 25850 external_mini_cluster.cc:1428] Setting key 931413981f3533115a6f6440c63915ea
Jun 19 20:39:09 dist-test-slave-0t40 krb5kdc[8702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901548, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:39:09.022428 8800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:09.013247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:35467 (local address 127.25.62.190:46749)
0619 20:39:09.013311 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:09.013314 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:09.013534 (+ 220us) server_negotiation.cc:407] Connection header received
0619 20:39:09.014071 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:09.014072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:09.014084 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:09.014130 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:39:09.015074 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.015580 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.016762 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.017096 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.017168 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:09.019583 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:39:09.019584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:39:09.019585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:39:09.019593 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:39:09.021028 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:09.021480 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:09.021481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:09.021482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:09.021542 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:09.021793 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:09.021794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:09.021794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:09.021908 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:39:09.021986 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:09.022254 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 20:39:09.022336 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:39:09.022619 9208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:09.013245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:35467)
0619 20:39:09.013407 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:09.013448 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:09.013964 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:09.014197 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:09.014201 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:09.014776 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:39:09.014975 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:09.014980 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.015685 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.015686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:09.016622 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:09.016625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.017194 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.017195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:09.017356 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:09.017984 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:39:09.017992 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:39:09.019490 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:39:09.021172 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:39:09.021179 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:39:09.021182 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:39:09.021398 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:39:09.021616 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:39:09.021617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:39:09.021618 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:39:09.021727 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:39:09.021978 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:39:09.021984 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:39:09.022123 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 20:39:09.022232 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 20:39:09.022389 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260619 20:39:09.023056 9206 heartbeater.cc:347] Connected to a master server at 127.25.62.190:46749
I20260619 20:39:09.023165 9206 heartbeater.cc:464] Registering TS with master...
I20260619 20:39:09.023356 9206 heartbeater.cc:511] Master 127.25.62.190:46749 requested a full tablet report, sending...
I20260619 20:39:09.023736 8747 ts_manager.cc:194] Registered new tserver with Master: 033c91b091ed4080b835cb06c5c976c5 (127.25.62.131:40907)
I20260619 20:39:09.024113 8747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:35467
I20260619 20:39:09.024871 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:39:09.031654 8800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:09.026192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41362 (local address 127.25.62.190:46749)
0619 20:39:09.026246 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:09.026248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:09.026358 (+ 110us) server_negotiation.cc:407] Connection header received
0619 20:39:09.026401 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:09.026402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:09.026410 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:09.026461 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:39:09.026936 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.027374 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.028049 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.028218 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.028264 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:09.029456 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:39:09.029457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:39:09.029457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:39:09.029462 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:39:09.030663 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:09.031033 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:09.031034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:09.031035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:09.031082 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:09.031241 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:09.031241 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:09.031242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:09.031364 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:39:09.031443 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:09.031543 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 20:39:09.031597 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 20:39:09.033335 8747 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:39:09.034797 8747 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:39:09.040973 9220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:09.037001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60322 (local address 127.25.62.131:40907)
0619 20:39:09.037209 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:09.037214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:09.037225 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:39:09.037263 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:09.037265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:09.037280 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:09.037382 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:39:09.037564 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.038474 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.039846 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.040564 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.040632 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:09.040685 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:09.040858 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 20:39:09.040897 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":109,"threads_started":1}
I20260619 20:39:09.040962 9218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:09.036842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:40907 (local address 127.0.0.1:60322)
0619 20:39:09.036984 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:09.037029 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:09.037117 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:09.037402 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:09.037404 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:09.037406 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:39:09.037472 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:09.037474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.038570 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.038571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:09.039764 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:09.039766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.040656 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.040657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:09.040747 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:09.040763 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 20:39:09.040833 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 20:39:09.040863 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":47,"threads_started":1}
I20260619 20:39:09.040962 9217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:09.036842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:44777 (local address 127.0.0.1:45958)
0619 20:39:09.037167 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:09.037177 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:09.037218 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:09.037652 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:09.037653 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:09.037656 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:39:09.037699 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:09.037701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.038987 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.038988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:09.040013 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:09.040015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.040696 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.040697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:09.040758 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:09.040765 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 20:39:09.040809 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 20:39:09.040861 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":2,"spinlock_wait_cycles":1408,"thread_start_us":82,"threads_started":1}
I20260619 20:39:09.041347 9219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:09.036952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45958 (local address 127.25.62.130:44777)
0619 20:39:09.037492 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:09.037495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:09.037502 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:39:09.037552 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:09.037554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:09.037569 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:09.037640 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:39:09.037780 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.038882 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.040100 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.040583 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.040997 (+ 414us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:09.041220 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:09.041254 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 20:39:09.041288 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":127,"threads_started":1}
I20260619 20:39:09.041563 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:09.037057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:39417 (local address 127.0.0.1:55102)
0619 20:39:09.037736 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:09.037746 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:09.037793 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:09.037995 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:09.037996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:09.037998 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:39:09.038036 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:09.038037 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.039190 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.039191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:09.040346 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:09.040349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.041344 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.041345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:09.041403 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:09.041407 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:39:09.041451 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 20:39:09.041489 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":65,"threads_started":1}
I20260619 20:39:09.041774 9222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:09.037256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55102 (local address 127.25.62.129:39417)
0619 20:39:09.037438 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:09.037442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:09.037849 (+ 407us) server_negotiation.cc:407] Connection header received
0619 20:39:09.037892 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:09.037894 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:09.037907 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:09.037981 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:39:09.038096 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.039090 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.040439 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.041186 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.041261 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:09.041312 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:09.041658 (+ 346us) server_negotiation.cc:299] Negotiation successful
0619 20:39:09.041700 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":86,"threads_started":1}
I20260619 20:39:09.042093 9140 tablet_service.cc:1467] Processing CreateTablet for tablet b18d493c063c408fb2f419aeb68e8a52 (DEFAULT_TABLE table=test-table [id=63beeafe698848e3a8c0925ec7dfe0b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:39:09.042071 9006 tablet_service.cc:1467] Processing CreateTablet for tablet b18d493c063c408fb2f419aeb68e8a52 (DEFAULT_TABLE table=test-table [id=63beeafe698848e3a8c0925ec7dfe0b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:39:09.042306 8872 tablet_service.cc:1467] Processing CreateTablet for tablet b18d493c063c408fb2f419aeb68e8a52 (DEFAULT_TABLE table=test-table [id=63beeafe698848e3a8c0925ec7dfe0b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:39:09.042413 9140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b18d493c063c408fb2f419aeb68e8a52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:39:09.042428 9006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b18d493c063c408fb2f419aeb68e8a52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:39:09.042583 8872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b18d493c063c408fb2f419aeb68e8a52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:39:09.044934 9223 tablet_bootstrap.cc:492] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5: Bootstrap starting.
I20260619 20:39:09.045691 9223 tablet_bootstrap.cc:654] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5: Neither blocks nor log segments found. Creating new log.
I20260619 20:39:09.045898 9225 tablet_bootstrap.cc:492] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69: Bootstrap starting.
I20260619 20:39:09.046348 9223 log.cc:824] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5: Log is configured to *not* fsync() on all Append() calls
I20260619 20:39:09.046365 9224 tablet_bootstrap.cc:492] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61: Bootstrap starting.
I20260619 20:39:09.046998 9224 tablet_bootstrap.cc:654] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61: Neither blocks nor log segments found. Creating new log.
I20260619 20:39:09.047078 9223 tablet_bootstrap.cc:492] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5: No bootstrap required, opened a new log
I20260619 20:39:09.047139 9223 ts_tablet_manager.cc:1397] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:39:09.047559 9224 log.cc:824] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61: Log is configured to *not* fsync() on all Append() calls
I20260619 20:39:09.048153 9225 tablet_bootstrap.cc:654] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69: Neither blocks nor log segments found. Creating new log.
I20260619 20:39:09.048400 9224 tablet_bootstrap.cc:492] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61: No bootstrap required, opened a new log
I20260619 20:39:09.048486 9224 ts_tablet_manager.cc:1397] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:39:09.048707 9225 log.cc:824] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69: Log is configured to *not* fsync() on all Append() calls
I20260619 20:39:09.049436 9223 raft_consensus.cc:348] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:09.049599 9223 raft_consensus.cc:374] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:39:09.049642 9223 raft_consensus.cc:729] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 033c91b091ed4080b835cb06c5c976c5, State: Initialized, Role: FOLLOWER
I20260619 20:39:09.049690 9225 tablet_bootstrap.cc:492] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69: No bootstrap required, opened a new log
I20260619 20:39:09.049758 9225 ts_tablet_manager.cc:1397] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 20:39:09.049753 9223 consensus_queue.cc:260] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:09.050035 9206 heartbeater.cc:503] Master 127.25.62.190:46749 was elected leader, sending a full tablet report...
I20260619 20:39:09.050035 9223 ts_tablet_manager.cc:1428] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:39:09.050639 9224 raft_consensus.cc:348] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:09.050931 9224 raft_consensus.cc:374] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:39:09.050976 9224 raft_consensus.cc:729] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb7e685f461044bb9c20a8a8838efe61, State: Initialized, Role: FOLLOWER
I20260619 20:39:09.051084 9224 consensus_queue.cc:260] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:09.051463 9224 ts_tablet_manager.cc:1428] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:39:09.051594 8938 heartbeater.cc:503] Master 127.25.62.190:46749 was elected leader, sending a full tablet report...
I20260619 20:39:09.052074 9225 raft_consensus.cc:348] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:09.052214 9225 raft_consensus.cc:374] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:39:09.052269 9225 raft_consensus.cc:729] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14d00dba0a6c4f8e9470116cefa36c69, State: Initialized, Role: FOLLOWER
I20260619 20:39:09.052399 9225 consensus_queue.cc:260] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:09.052698 9225 ts_tablet_manager.cc:1428] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:39:09.052748 9072 heartbeater.cc:503] Master 127.25.62.190:46749 was elected leader, sending a full tablet report...
W20260619 20:39:09.089196 9073 tablet.cc:2369] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:39:09.190295 8939 tablet.cc:2369] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:39:09.263005 9207 tablet.cc:2369] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:39:09.276942 9230 raft_consensus.cc:482] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:39:09.277123 9230 raft_consensus.cc:504] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:09.277465 9230 leader_election.cc:288] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 033c91b091ed4080b835cb06c5c976c5 (127.25.62.131:40907), 14d00dba0a6c4f8e9470116cefa36c69 (127.25.62.130:44777)
I20260619 20:39:09.280378 9219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:09.277830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:43611 (local address 127.25.62.130:44777)
0619 20:39:09.277884 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:09.277887 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:09.277895 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:39:09.277924 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:09.277925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:09.277937 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:09.277987 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:39:09.278161 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.278641 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.279658 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.280036 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.280121 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:09.280128 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:09.280277 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 20:39:09.280309 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 20:39:09.280378 9220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:09.277687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:53951 (local address 127.25.62.131:40907)
0619 20:39:09.277750 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:09.277752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:09.277830 (+ 78us) server_negotiation.cc:407] Connection header received
0619 20:39:09.277943 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:09.277944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:09.277952 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:09.278021 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:39:09.278194 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.278655 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.279642 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.280011 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.280104 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:09.280113 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:09.280270 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 20:39:09.280314 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:39:09.280392 9232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:09.277630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:40907 (local address 127.25.62.129:53951)
0619 20:39:09.277752 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:09.277762 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:09.277843 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:09.278025 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:09.278026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:09.278028 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:39:09.278121 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:09.278123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.278738 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.278739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:09.279550 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:09.279553 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.280104 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.280105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:09.280183 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:09.280192 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 20:39:09.280254 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 20:39:09.280297 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":28,"thread_start_us":57,"threads_started":1}
I20260619 20:39:09.280405 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:09.277665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:44777 (local address 127.25.62.129:43611)
0619 20:39:09.277792 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:09.277803 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:09.277855 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:09.278001 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:09.278002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:09.278004 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:39:09.278084 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:09.278086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.278725 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.278726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:09.279573 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:09.279576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.280120 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.280121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:09.280199 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:09.280202 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:39:09.280270 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 20:39:09.280309 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":1,"thread_start_us":72,"threads_started":1}
I20260619 20:39:09.280702 9229 raft_consensus.cc:482] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:39:09.280747 9160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18d493c063c408fb2f419aeb68e8a52" candidate_uuid: "eb7e685f461044bb9c20a8a8838efe61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "033c91b091ed4080b835cb06c5c976c5" is_pre_election: true
I20260619 20:39:09.280797 9229 raft_consensus.cc:504] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:09.280961 9160 raft_consensus.cc:2454] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb7e685f461044bb9c20a8a8838efe61 in term 0.
I20260619 20:39:09.281008 9026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18d493c063c408fb2f419aeb68e8a52" candidate_uuid: "eb7e685f461044bb9c20a8a8838efe61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14d00dba0a6c4f8e9470116cefa36c69" is_pre_election: true
I20260619 20:39:09.281185 9026 raft_consensus.cc:2454] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb7e685f461044bb9c20a8a8838efe61 in term 0.
I20260619 20:39:09.281242 8826 leader_election.cc:302] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 033c91b091ed4080b835cb06c5c976c5, eb7e685f461044bb9c20a8a8838efe61; no voters:
I20260619 20:39:09.281474 9230 raft_consensus.cc:2781] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:39:09.281545 9230 raft_consensus.cc:482] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:39:09.281571 9230 raft_consensus.cc:3037] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:39:09.281962 9229 leader_election.cc:288] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14d00dba0a6c4f8e9470116cefa36c69 (127.25.62.130:44777), eb7e685f461044bb9c20a8a8838efe61 (127.25.62.129:39417)
I20260619 20:39:09.282480 9230 raft_consensus.cc:504] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:09.282630 9230 leader_election.cc:288] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 1 election: Requested vote from peers 033c91b091ed4080b835cb06c5c976c5 (127.25.62.131:40907), 14d00dba0a6c4f8e9470116cefa36c69 (127.25.62.130:44777)
I20260619 20:39:09.282829 9160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18d493c063c408fb2f419aeb68e8a52" candidate_uuid: "eb7e685f461044bb9c20a8a8838efe61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "033c91b091ed4080b835cb06c5c976c5"
I20260619 20:39:09.282837 9026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18d493c063c408fb2f419aeb68e8a52" candidate_uuid: "eb7e685f461044bb9c20a8a8838efe61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14d00dba0a6c4f8e9470116cefa36c69"
I20260619 20:39:09.282898 9160 raft_consensus.cc:3037] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:39:09.282898 9026 raft_consensus.cc:3037] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:39:09.283725 9160 raft_consensus.cc:2454] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb7e685f461044bb9c20a8a8838efe61 in term 1.
I20260619 20:39:09.283725 9026 raft_consensus.cc:2454] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb7e685f461044bb9c20a8a8838efe61 in term 1.
I20260619 20:39:09.283902 8827 leader_election.cc:302] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14d00dba0a6c4f8e9470116cefa36c69, eb7e685f461044bb9c20a8a8838efe61; no voters:
I20260619 20:39:09.284016 9230 raft_consensus.cc:2781] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:39:09.284161 9230 raft_consensus.cc:686] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [term 1 LEADER]: Becoming Leader. State: Replica: eb7e685f461044bb9c20a8a8838efe61, State: Running, Role: LEADER
I20260619 20:39:09.284253 9230 consensus_queue.cc:237] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:09.284664 9208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:09.281313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:44777 (local address 127.25.62.131:60845)
0619 20:39:09.281394 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:09.281406 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:09.281459 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:09.281600 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:09.281601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:09.281603 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:39:09.281659 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:09.281660 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.282431 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.282432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:09.283563 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:09.283566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.284397 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.284398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:09.284497 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:09.284500 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:39:09.284557 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 20:39:09.284600 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260619 20:39:09.284679 9219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:09.281368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:60845 (local address 127.25.62.130:44777)
0619 20:39:09.281410 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:09.281413 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:09.281445 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:39:09.281508 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:09.281509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:09.281517 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:09.281563 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:39:09.281721 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.282340 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.283655 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.284178 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.284237 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:09.284243 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:09.284573 (+ 330us) server_negotiation.cc:299] Negotiation successful
0619 20:39:09.284618 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:39:09.284685 9222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:09.281815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:54413 (local address 127.25.62.129:39417)
0619 20:39:09.281851 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:09.281853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:09.281860 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:39:09.281880 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:09.281881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:09.281889 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:09.281933 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:39:09.282062 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.282565 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.283833 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.284357 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.284398 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:09.284405 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:09.284600 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 20:39:09.284632 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:39:09.284894 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:09.281637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:39417 (local address 127.25.62.131:54413)
0619 20:39:09.281787 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:09.281797 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:09.281836 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:09.281944 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:09.281944 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:09.281946 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:39:09.282009 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:09.282010 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.282654 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.282655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:09.283745 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:09.283747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.284457 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.284458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:09.284521 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:09.284524 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:39:09.284777 (+ 253us) client_negotiation.cc:240] Negotiation successful
0619 20:39:09.284821 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260619 20:39:09.284891 9026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18d493c063c408fb2f419aeb68e8a52" candidate_uuid: "033c91b091ed4080b835cb06c5c976c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14d00dba0a6c4f8e9470116cefa36c69" is_pre_election: true
I20260619 20:39:09.284977 9026 raft_consensus.cc:2379] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 033c91b091ed4080b835cb06c5c976c5 in current term 1: Already voted for candidate eb7e685f461044bb9c20a8a8838efe61 in this term.
I20260619 20:39:09.285144 8892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b18d493c063c408fb2f419aeb68e8a52" candidate_uuid: "033c91b091ed4080b835cb06c5c976c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb7e685f461044bb9c20a8a8838efe61" is_pre_election: true
I20260619 20:39:09.285480 9095 leader_election.cc:302] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 033c91b091ed4080b835cb06c5c976c5; no voters: 14d00dba0a6c4f8e9470116cefa36c69, eb7e685f461044bb9c20a8a8838efe61
I20260619 20:39:09.285593 9229 raft_consensus.cc:2726] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 20:39:09.285614 8747 catalog_manager.cc:5495] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb7e685f461044bb9c20a8a8838efe61 (127.25.62.129). New cstate: current_term: 1 leader_uuid: "eb7e685f461044bb9c20a8a8838efe61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } health_report { overall_health: HEALTHY } } }
I20260619 20:39:09.327181 9222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:09.324684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55112 (local address 127.25.62.129:39417)
0619 20:39:09.324764 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:09.324766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:09.324774 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:39:09.324811 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:09.324812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:09.324830 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:09.324897 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:39:09.325024 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.325572 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.326336 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:09.326536 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:09.326627 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:09.326822 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:39:09.326927 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:39:09.327020 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:09.327064 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 20:39:09.327086 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:39:09.328940 9026 raft_consensus.cc:1261] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69 [term 1 FOLLOWER]: Refusing update from remote peer eb7e685f461044bb9c20a8a8838efe61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:39:09.328959 9160 raft_consensus.cc:1261] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5 [term 1 FOLLOWER]: Refusing update from remote peer eb7e685f461044bb9c20a8a8838efe61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:39:09.329285 9230 consensus_queue.cc:1035] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:39:09.329495 9235 consensus_queue.cc:1035] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:39:09.331835 9238 mvcc.cc:205] Tried to move back new op lower bound from 7298668746048716800 to 7298668745868955648. Current Snapshot: MvccSnapshot[applied={T|T < 7298668746048716800}]
I20260619 20:39:09.332108 9239 mvcc.cc:205] Tried to move back new op lower bound from 7298668746048716800 to 7298668745868955648. Current Snapshot: MvccSnapshot[applied={T|T < 7298668746048716800}]
I20260619 20:39:09.332798 9240 mvcc.cc:205] Tried to move back new op lower bound from 7298668746048716800 to 7298668745868955648. Current Snapshot: MvccSnapshot[applied={T|T < 7298668746048716800 or (T in {7298668746048716800})}]
I20260619 20:39:09.334133 8747 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:39:09.334223 8747 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:39:09.335000 8747 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501: Sending DeleteTablet for 3 replicas of tablet b18d493c063c408fb2f419aeb68e8a52
I20260619 20:39:09.335325 8872 tablet_service.cc:1514] Processing DeleteTablet for tablet b18d493c063c408fb2f419aeb68e8a52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:39:09 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:55102
I20260619 20:39:09.335343 9006 tablet_service.cc:1514] Processing DeleteTablet for tablet b18d493c063c408fb2f419aeb68e8a52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:39:09 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:45958
I20260619 20:39:09.335361 9140 tablet_service.cc:1514] Processing DeleteTablet for tablet b18d493c063c408fb2f419aeb68e8a52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:39:09 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:60322
I20260619 20:39:09.335536 9249 tablet_replica.cc:330] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69: stopping tablet replica
I20260619 20:39:09.335633 9249 raft_consensus.cc:2229] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:39:09.335665 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 8809
I20260619 20:39:09.335741 9249 raft_consensus.cc:2258] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:39:09.336527 9249 ts_tablet_manager.cc:1905] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:39:09.338490 9249 ts_tablet_manager.cc:1918] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:39:09.338567 9249 log.cc:1196] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/b18d493c063c408fb2f419aeb68e8a52
I20260619 20:39:09.338835 9249 ts_tablet_manager.cc:1939] T b18d493c063c408fb2f419aeb68e8a52 P 14d00dba0a6c4f8e9470116cefa36c69: Deleting consensus metadata
I20260619 20:39:09.339110 9250 tablet_replica.cc:330] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5: stopping tablet replica
I20260619 20:39:09.339212 9250 raft_consensus.cc:2229] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:39:09.339293 8733 catalog_manager.cc:4841] TS 14d00dba0a6c4f8e9470116cefa36c69 (127.25.62.130:44777): tablet b18d493c063c408fb2f419aeb68e8a52 (table test-table [id=63beeafe698848e3a8c0925ec7dfe0b4]) successfully deleted
I20260619 20:39:09.339327 9250 raft_consensus.cc:2258] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:39:09.339753 9250 ts_tablet_manager.cc:1905] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:39:09.341007 9250 ts_tablet_manager.cc:1918] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:39:09.341135 9250 log.cc:1196] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/b18d493c063c408fb2f419aeb68e8a52
I20260619 20:39:09.341662 9250 ts_tablet_manager.cc:1939] T b18d493c063c408fb2f419aeb68e8a52 P 033c91b091ed4080b835cb06c5c976c5: Deleting consensus metadata
I20260619 20:39:09.342051 8732 catalog_manager.cc:4841] TS 033c91b091ed4080b835cb06c5c976c5 (127.25.62.131:40907): tablet b18d493c063c408fb2f419aeb68e8a52 (table test-table [id=63beeafe698848e3a8c0925ec7dfe0b4]) successfully deleted
W20260619 20:39:09.342875 8732 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.62.129:39417 (error 108)
I20260619 20:39:09.343168 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 8943
I20260619 20:39:09.343365 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:09.343256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:39417 (local address 127.0.0.1:55124)
0619 20:39:09.343292 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:09.343329 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.129:39417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260619 20:39:09.344285 8732 catalog_manager.cc:4584] TS eb7e685f461044bb9c20a8a8838efe61 (127.25.62.129:39417): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b18d493c063c408fb2f419aeb68e8a52: Network error: Client connection negotiation failed: client connection to 127.25.62.129:39417: connect: Connection refused (error 111)
I20260619 20:39:09.347909 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 9077
I20260619 20:39:09.352301 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 8715
I20260619 20:39:29.357622 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:46749
--webserver_interface=127.25.62.190
--webserver_port=39817
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:46749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:39:29.437479 9252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:39:29.437680 9252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:39:29.437702 9252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:39:29.438843 9252 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 20:39:29.438880 9252 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 20:39:29.438894 9252 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 20:39:29.438906 9252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:39:29.438941 9252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:39:29.438974 9252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:39:29.438992 9252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:39:29.439009 9252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:39:29.440491 9252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:46749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:46749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=39817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:39:29.440733 9252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:39:29.440963 9252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:39:29.449331 9259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:39:29.449369 9252 server_base.cc:1034] running on GCE node
W20260619 20:39:29.449463 9258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:39:29.449366 9261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:39:29.449808 9252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:39:29.450107 9252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:39:29.451263 9252 hybrid_clock.cc:648] HybridClock initialized: now 1781901569451252 us; error 37 us; skew 500 ppm
Jun 19 20:39:29 dist-test-slave-0t40 krb5kdc[8702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901569, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:39:29.453891 9252 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:39:29.454346 9252 webserver.cc:460] Webserver started at http://127.25.62.190:39817/ using document root <none> and password file <none>
I20260619 20:39:29.454595 9252 fs_manager.cc:359] Metadata directory not provided
I20260619 20:39:29.454674 9252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:39:29.456012 9252 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:39:29.456634 9268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:29.456820 9252 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:39:29.456918 9252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "9f6af0c094d2409e8a9f27895b305501"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "4231f42bd033cbe7a27d55b5f3d64fc7"
server_key_iv: "c9f269a8104bb4f75c68e0d2ce48af97"
server_key_version: "encryptionkey@0"
I20260619 20:39:29.457342 9252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:39:29.472186 9252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:39:29.474467 9252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:39:29.474633 9252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:39:29.478598 9252 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:46749
I20260619 20:39:29.478648 9320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:46749 every 8 connection(s)
I20260619 20:39:29.479038 9252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:39:29.480556 9321 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:39:29.481288 9321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501: Bootstrap starting.
I20260619 20:39:29.482892 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 9252
I20260619 20:39:29.483281 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:39417
--local_ip_for_outbound_sockets=127.25.62.129
--tserver_master_addrs=127.25.62.190:46749
--webserver_port=37511
--webserver_interface=127.25.62.129
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:39:29.490600 9321 log.cc:824] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501: Log is configured to *not* fsync() on all Append() calls
I20260619 20:39:29.492630 9321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:39:29.492965 9321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501: Bootstrap complete.
I20260619 20:39:29.495281 9321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6af0c094d2409e8a9f27895b305501" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46749 } }
I20260619 20:39:29.495515 9321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f6af0c094d2409e8a9f27895b305501, State: Initialized, Role: FOLLOWER
I20260619 20:39:29.495693 9321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6af0c094d2409e8a9f27895b305501" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46749 } }
I20260619 20:39:29.495764 9321 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:39:29.495819 9321 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:39:29.495872 9321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 1 FOLLOWER]: Advancing to term 2
I20260619 20:39:29.497006 9321 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6af0c094d2409e8a9f27895b305501" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46749 } }
I20260619 20:39:29.497165 9321 leader_election.cc:302] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f6af0c094d2409e8a9f27895b305501; no voters:
I20260619 20:39:29.497365 9321 leader_election.cc:288] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 20:39:29.497404 9325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 20:39:29.497524 9325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 2 LEADER]: Becoming Leader. State: Replica: 9f6af0c094d2409e8a9f27895b305501, State: Running, Role: LEADER
I20260619 20:39:29.497645 9321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:39:29.497689 9325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6af0c094d2409e8a9f27895b305501" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46749 } }
I20260619 20:39:29.498433 9326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9f6af0c094d2409e8a9f27895b305501" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6af0c094d2409e8a9f27895b305501" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46749 } } }
I20260619 20:39:29.498449 9327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f6af0c094d2409e8a9f27895b305501. Latest consensus state: current_term: 2 leader_uuid: "9f6af0c094d2409e8a9f27895b305501" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6af0c094d2409e8a9f27895b305501" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46749 } } }
I20260619 20:39:29.498543 9326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [sys.catalog]: This master's current role is: LEADER
I20260619 20:39:29.498562 9327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [sys.catalog]: This master's current role is: LEADER
I20260619 20:39:29.498944 9334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:39:29.499886 9334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:39:29.499993 9334 catalog_manager.cc:1259] Loaded cluster ID: c3518f7e531849e7a5f827d6bc0af70d
I20260619 20:39:29.500030 9334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:39:29.501228 9334 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:39:29.501420 9334 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501: Loaded TSK: 0
I20260619 20:39:29.514889 9334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501: Generated new TSK 1
I20260619 20:39:29.515012 9334 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 20:39:29.567312 9323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:39:29.567541 9323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:39:29.567584 9323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:39:29.569027 9323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:39:29.569154 9323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:39:29.569200 9323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:39:29.570782 9323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:39417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=37511
--tserver_master_addrs=127.25.62.190:46749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:39:29.571043 9323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:39:29.571264 9323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:39:29.576144 9349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:39:29.576201 9348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:39:29.576356 9323 server_base.cc:1034] running on GCE node
W20260619 20:39:29.576151 9351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:39:29.576689 9323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:39:29.576965 9323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:39:29.578152 9323 hybrid_clock.cc:648] HybridClock initialized: now 1781901569578131 us; error 41 us; skew 500 ppm
Jun 19 20:39:29 dist-test-slave-0t40 krb5kdc[8702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901569, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:39:29.580868 9323 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:39:29.581461 9323 webserver.cc:460] Webserver started at http://127.25.62.129:37511/ using document root <none> and password file <none>
I20260619 20:39:29.581739 9323 fs_manager.cc:359] Metadata directory not provided
I20260619 20:39:29.581830 9323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:39:29.583683 9323 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:39:29.584506 9358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:29.584824 9323 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:39:29.584928 9323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "eb7e685f461044bb9c20a8a8838efe61"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "27cd64a84fac91b22f72739059f2c960"
server_key_iv: "584a70992d78548f10a5524a0c7203b7"
server_key_version: "encryptionkey@0"
I20260619 20:39:29.585410 9323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:39:29.599303 9323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:39:29.602121 9323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:39:29.602299 9323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:39:29.602547 9323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:39:29.603077 9365 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 20:39:29.604516 9323 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 20:39:29.604583 9323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 20:39:29.604621 9323 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 20:39:29.605641 9323 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 20:39:29.605700 9323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260619 20:39:29.605782 9365 tablet_bootstrap.cc:492] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61: Bootstrap starting.
I20260619 20:39:29.613555 9323 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:39417
I20260619 20:39:29.614092 9323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:39:29.616636 9365 log.cc:824] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61: Log is configured to *not* fsync() on all Append() calls
I20260619 20:39:29.618759 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 9323
I20260619 20:39:29.619328 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:44777
--local_ip_for_outbound_sockets=127.25.62.130
--tserver_master_addrs=127.25.62.190:46749
--webserver_port=33683
--webserver_interface=127.25.62.130
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:39:29.619653 9365 tablet_bootstrap.cc:492] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:39:29.619997 9365 tablet_bootstrap.cc:492] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61: Bootstrap complete.
I20260619 20:39:29.621275 9365 ts_tablet_manager.cc:1397] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.015s
I20260619 20:39:29.622788 9365 raft_consensus.cc:348] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:29.623171 9365 raft_consensus.cc:729] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb7e685f461044bb9c20a8a8838efe61, State: Initialized, Role: FOLLOWER
I20260619 20:39:29.623384 9365 consensus_queue.cc:260] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:29.623757 9365 ts_tablet_manager.cc:1428] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
Jun 19 20:39:29 dist-test-slave-0t40 krb5kdc[8702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901569, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:39:29.625241 9472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:39417 every 8 connection(s)
I20260619 20:39:29.629398 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:29.615003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:37781)
0619 20:39:29.615231 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:29.615288 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:29.616060 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:29.617416 (+ 1356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:29.617419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:29.617731 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:39:29.617922 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:29.617927 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.619783 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.619784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:29.621264 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:29.621269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.622218 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.622219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:29.622407 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:29.622998 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:39:29.623006 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:39:29.624562 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:39:29.627350 (+ 2788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:39:29.627357 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:39:29.627359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:39:29.627676 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:39:29.628000 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:39:29.628001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:39:29.628001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:39:29.628143 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:39:29.628668 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:39:29.628674 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:39:29.628831 (+ 157us) client_negotiation.cc:769] Sending connection context
0619 20:39:29.628964 (+ 133us) client_negotiation.cc:240] Negotiation successful
0619 20:39:29.629181 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":108,"threads_started":1}
I20260619 20:39:29.629542 9476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:29.615370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:37781 (local address 127.25.62.190:46749)
0619 20:39:29.616949 (+ 1579us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:29.616958 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:29.617003 (+ 45us) server_negotiation.cc:407] Connection header received
0619 20:39:29.617160 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:29.617169 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:29.617231 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:29.617384 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:39:29.618146 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.619625 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.621397 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.622073 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.622697 (+ 624us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:29.624695 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:39:29.624699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:39:29.624700 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:39:29.624712 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:39:29.627246 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:29.627779 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:29.627783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:29.627783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:29.627913 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:29.628227 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:29.628228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:29.628228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:29.628577 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:39:29.628677 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:29.629095 (+ 418us) server_negotiation.cc:299] Negotiation successful
0619 20:39:29.629273 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1524,"thread_start_us":86,"threads_started":1}
I20260619 20:39:29.630589 9473 heartbeater.cc:347] Connected to a master server at 127.25.62.190:46749
I20260619 20:39:29.630699 9473 heartbeater.cc:464] Registering TS with master...
I20260619 20:39:29.630889 9473 heartbeater.cc:511] Master 127.25.62.190:46749 requested a full tablet report, sending...
I20260619 20:39:29.631728 9285 ts_manager.cc:194] Registered new tserver with Master: eb7e685f461044bb9c20a8a8838efe61 (127.25.62.129:39417)
I20260619 20:39:29.633103 9285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:37781
I20260619 20:39:29.634117 9473 heartbeater.cc:503] Master 127.25.62.190:46749 was elected leader, sending a full tablet report...
I20260619 20:39:29.641685 9480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:29.632432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:39417 (local address 127.0.0.1:43416)
0619 20:39:29.637182 (+ 4750us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:29.637236 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:29.637310 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:29.637659 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:29.637662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:29.637665 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:39:29.637748 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:29.637749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.639020 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.639021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:29.640370 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:29.640373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.641341 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.641343 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:29.641451 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:29.641466 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 20:39:29.641544 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 20:39:29.641596 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4712,"thread_start_us":117,"threads_started":1}
I20260619 20:39:29.641690 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:29.633140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43416 (local address 127.25.62.129:39417)
0619 20:39:29.637456 (+ 4316us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:29.637460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:29.637470 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:39:29.637505 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:29.637507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:29.637523 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:29.637634 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:39:29.637836 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.638923 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.640464 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.641234 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.641319 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:29.641372 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:29.641568 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 20:39:29.641612 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4281,"thread_start_us":90,"threads_started":1}
I20260619 20:39:29.642237 9405 tablet_service.cc:1514] Processing DeleteTablet for tablet b18d493c063c408fb2f419aeb68e8a52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:39:09 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:43416
I20260619 20:39:29.642853 9482 tablet_replica.cc:330] stopping tablet replica
I20260619 20:39:29.643007 9482 raft_consensus.cc:2229] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:39:29.643085 9482 raft_consensus.cc:2258] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:39:29.643417 9482 ts_tablet_manager.cc:1905] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:39:29.645928 9482 ts_tablet_manager.cc:1918] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:39:29.646003 9482 log.cc:1196] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/wal/wals/b18d493c063c408fb2f419aeb68e8a52
I20260619 20:39:29.646221 9482 ts_tablet_manager.cc:1939] T b18d493c063c408fb2f419aeb68e8a52 P eb7e685f461044bb9c20a8a8838efe61: Deleting consensus metadata
I20260619 20:39:29.646816 9270 catalog_manager.cc:4841] TS eb7e685f461044bb9c20a8a8838efe61 (127.25.62.129:39417): tablet b18d493c063c408fb2f419aeb68e8a52 (table test-table [id=63beeafe698848e3a8c0925ec7dfe0b4]) successfully deleted
W20260619 20:39:29.708353 9478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:39:29.708540 9478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:39:29.708562 9478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:39:29.709741 9478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:39:29.709781 9478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:39:29.709806 9478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:39:29.711166 9478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:44777
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=33683
--tserver_master_addrs=127.25.62.190:46749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:39:29.711393 9478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:39:29.711599 9478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:39:29.716552 9490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:39:29.716573 9487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:39:29.716594 9488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:39:29.716651 9478 server_base.cc:1034] running on GCE node
I20260619 20:39:29.717177 9478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:39:29.717458 9478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:39:29.718636 9478 hybrid_clock.cc:648] HybridClock initialized: now 1781901569718624 us; error 39 us; skew 500 ppm
Jun 19 20:39:29 dist-test-slave-0t40 krb5kdc[8702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901569, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:39:29.721150 9478 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:39:29.721621 9478 webserver.cc:460] Webserver started at http://127.25.62.130:33683/ using document root <none> and password file <none>
I20260619 20:39:29.721853 9478 fs_manager.cc:359] Metadata directory not provided
I20260619 20:39:29.721928 9478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:39:29.723243 9478 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:39:29.723857 9497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:29.724027 9478 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:39:29.724119 9478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "14d00dba0a6c4f8e9470116cefa36c69"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "0f7dde673f0c82b0b72f05ce7dbd33b6"
server_key_iv: "6060b90292c99ab92382dbac44e2c5f6"
server_key_version: "encryptionkey@0"
I20260619 20:39:29.724458 9478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:39:29.739310 9478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:39:29.741284 9478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:39:29.741443 9478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:39:29.741677 9478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:39:29.741988 9478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:39:29.742039 9478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:29.742081 9478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:39:29.742110 9478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:29.747810 9478 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:44777
I20260619 20:39:29.747857 9610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:44777 every 8 connection(s)
I20260619 20:39:29.748222 9478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
Jun 19 20:39:29 dist-test-slave-0t40 krb5kdc[8702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901569, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:39:29.756940 9476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:29.748991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:52259 (local address 127.25.62.190:46749)
0619 20:39:29.749087 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:29.749089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:29.749257 (+ 168us) server_negotiation.cc:407] Connection header received
0619 20:39:29.749726 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:29.749727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:29.749736 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:29.749777 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:39:29.750693 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.751212 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.752146 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.752352 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.752403 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:29.754516 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:39:29.754517 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:39:29.754517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:39:29.754524 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:39:29.755735 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:29.756174 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:29.756175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:29.756175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:29.756229 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:29.756440 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:29.756441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:29.756441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:29.756542 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:39:29.756592 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:29.756796 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 20:39:29.756874 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 20:39:29.757112 9613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:29.748957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:52259)
0619 20:39:29.749130 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:29.749170 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:29.749646 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:29.749839 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:29.749842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:29.750409 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:39:29.750597 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:29.750602 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.751290 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.751290 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:29.752074 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:29.752076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.752418 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.752419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:29.752515 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:29.752952 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:39:29.752959 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:39:29.754424 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:39:29.755807 (+ 1383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:39:29.755814 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:39:29.755816 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:39:29.756102 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:39:29.756286 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:39:29.756287 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:39:29.756287 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:39:29.756391 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:39:29.756592 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:39:29.756596 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:39:29.756706 (+ 110us) client_negotiation.cc:769] Sending connection context
0619 20:39:29.756785 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 20:39:29.756903 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":109,"threads_started":1}
I20260619 20:39:29.757663 9611 heartbeater.cc:347] Connected to a master server at 127.25.62.190:46749
I20260619 20:39:29.757817 9611 heartbeater.cc:464] Registering TS with master...
I20260619 20:39:29.757977 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 9478
I20260619 20:39:29.758008 9611 heartbeater.cc:511] Master 127.25.62.190:46749 requested a full tablet report, sending...
I20260619 20:39:29.758282 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:40907
--local_ip_for_outbound_sockets=127.25.62.131
--tserver_master_addrs=127.25.62.190:46749
--webserver_port=44565
--webserver_interface=127.25.62.131
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:39:29.758390 9285 ts_manager.cc:194] Registered new tserver with Master: 14d00dba0a6c4f8e9470116cefa36c69 (127.25.62.130:44777)
I20260619 20:39:29.758893 9285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:52259
W20260619 20:39:29.838088 9614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:39:29.838317 9614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:39:29.838342 9614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:39:29.839440 9614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:39:29.839478 9614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:39:29.839505 9614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:39:29.840911 9614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:40907
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=44565
--tserver_master_addrs=127.25.62.190:46749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:39:29.841181 9614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:39:29.841420 9614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:39:29.846642 9620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:39:29.846657 9619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:39:29.846689 9614 server_base.cc:1034] running on GCE node
W20260619 20:39:29.846642 9622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:39:29.847126 9614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:39:29.847368 9614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:39:29.848519 9614 hybrid_clock.cc:648] HybridClock initialized: now 1781901569848511 us; error 33 us; skew 500 ppm
Jun 19 20:39:29 dist-test-slave-0t40 krb5kdc[8702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901569, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:39:29.850973 9614 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:39:29.851394 9614 webserver.cc:460] Webserver started at http://127.25.62.131:44565/ using document root <none> and password file <none>
I20260619 20:39:29.851608 9614 fs_manager.cc:359] Metadata directory not provided
I20260619 20:39:29.851684 9614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:39:29.853008 9614 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:39:29.853618 9629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:29.853766 9614 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:29.853857 9614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "033c91b091ed4080b835cb06c5c976c5"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "b93e39b2351f193b70454e6aec133fc0"
server_key_iv: "4e2924017a5500ffdfd132f0b04902ba"
server_key_version: "encryptionkey@0"
I20260619 20:39:29.854200 9614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:39:29.884372 9614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:39:29.886205 9614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:39:29.886345 9614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:39:29.886582 9614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:39:29.886900 9614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:39:29.886934 9614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:29.886957 9614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:39:29.886970 9614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:29.892782 9614 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:40907
I20260619 20:39:29.892829 9742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:40907 every 8 connection(s)
I20260619 20:39:29.893177 9614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:39:29.893764 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 9614
Jun 19 20:39:29 dist-test-slave-0t40 krb5kdc[8702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901569, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:39:29.901424 9746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:29.894134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39798 (local address 127.25.62.190:46749)
0619 20:39:29.894237 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:29.894239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:29.894388 (+ 149us) server_negotiation.cc:407] Connection header received
0619 20:39:29.894422 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:29.894423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:29.894438 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:29.894524 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:39:29.895072 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.895888 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.896699 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.897025 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.897120 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:29.898528 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:39:29.898530 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:39:29.898530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:39:29.898538 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:39:29.900162 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:29.900529 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:29.900529 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:29.900530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:29.900579 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:29.900778 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:29.900778 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:29.900779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:29.900912 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:39:29.901094 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:29.901157 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 20:39:29.901286 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":55,"threads_started":1}
I20260619 20:39:29.903646 9476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:29.893866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:52145 (local address 127.25.62.190:46749)
0619 20:39:29.893911 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:29.893913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:29.894185 (+ 272us) server_negotiation.cc:407] Connection header received
0619 20:39:29.894825 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:29.894826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:29.894837 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:29.894889 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:39:29.896020 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.896459 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.897769 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.898021 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.898099 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:29.900695 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:39:29.900696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:39:29.900696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:39:29.900701 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:39:29.902334 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:29.902726 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:29.902734 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:29.902734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:29.902791 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:29.903026 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:29.903027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:29.903028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:29.903140 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:39:29.903207 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:29.903483 (+ 276us) server_negotiation.cc:299] Negotiation successful
0619 20:39:29.903559 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:39:29.903828 9745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:29.893884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:52145)
0619 20:39:29.894063 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:29.894104 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:29.894725 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:29.894960 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:29.894964 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:29.895696 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:39:29.895951 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:29.895959 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.896543 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.896544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:29.897690 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:29.897693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.898110 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.898111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:29.898248 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:29.898711 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:39:29.898718 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:39:29.900599 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:39:29.902417 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:39:29.902423 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:39:29.902425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:39:29.902640 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:39:29.902852 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:39:29.902853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:39:29.902853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:39:29.902963 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:39:29.903213 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:39:29.903218 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:39:29.903354 (+ 136us) client_negotiation.cc:769] Sending connection context
0619 20:39:29.903451 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 20:39:29.903604 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":111,"threads_started":1}
I20260619 20:39:29.904268 9743 heartbeater.cc:347] Connected to a master server at 127.25.62.190:46749
I20260619 20:39:29.904359 9743 heartbeater.cc:464] Registering TS with master...
I20260619 20:39:29.904523 9743 heartbeater.cc:511] Master 127.25.62.190:46749 requested a full tablet report, sending...
I20260619 20:39:29.904865 9285 ts_manager.cc:194] Registered new tserver with Master: 033c91b091ed4080b835cb06c5c976c5 (127.25.62.131:40907)
I20260619 20:39:29.905238 9285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:52145
I20260619 20:39:29.912958 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:39:29.915812 9476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:29.913354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39810 (local address 127.25.62.190:46749)
0619 20:39:29.913430 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:29.913432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:29.913483 (+ 51us) server_negotiation.cc:407] Connection header received
0619 20:39:29.913550 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:29.913551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:29.913584 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:29.913662 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:39:29.913768 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.914237 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.915096 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.915282 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.915349 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:29.915531 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:39:29.915631 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:39:29.915717 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:29.915734 (+ 17us) server_negotiation.cc:299] Negotiation successful
0619 20:39:29.915751 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 20:39:29.916109 9285 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:39810:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:39:29.917826 9285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:39:29.920065 9405 tablet_service.cc:1467] Processing CreateTablet for tablet d64f7ae1e60b44e687b8b1e8065df3a5 (DEFAULT_TABLE table=test-table [id=b88b491207b74cbaa8dbef94b8cce753]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:39:29.920305 9405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d64f7ae1e60b44e687b8b1e8065df3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:39:29.921919 9365 tablet_bootstrap.cc:492] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61: Bootstrap starting.
I20260619 20:39:29.922258 9365 tablet_bootstrap.cc:654] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61: Neither blocks nor log segments found. Creating new log.
I20260619 20:39:29.923513 9480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:29.919600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:44777 (local address 127.0.0.1:32968)
0619 20:39:29.919652 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:29.919663 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:29.919707 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:29.920037 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:29.920039 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:29.920040 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:39:29.920084 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:29.920085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.921372 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.921374 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:29.922354 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:29.922357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.923301 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.923302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:29.923355 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:29.923358 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:39:29.923423 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 20:39:29.923461 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 20:39:29.923563 9750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:29.919709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32968 (local address 127.25.62.130:44777)
0619 20:39:29.919870 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:29.919875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:29.919885 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:39:29.919926 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:29.919928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:29.919944 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:29.920026 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:39:29.920193 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.921265 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.922470 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.923199 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.923277 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:29.923336 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:29.923438 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 20:39:29.923476 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
I20260619 20:39:29.923954 9365 tablet_bootstrap.cc:492] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61: No bootstrap required, opened a new log
I20260619 20:39:29.924010 9365 ts_tablet_manager.cc:1397] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 20:39:29.924132 9365 raft_consensus.cc:348] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:29.924216 9365 raft_consensus.cc:374] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:39:29.924270 9365 raft_consensus.cc:729] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb7e685f461044bb9c20a8a8838efe61, State: Initialized, Role: FOLLOWER
I20260619 20:39:29.924328 9365 consensus_queue.cc:260] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:29.924420 9365 ts_tablet_manager.cc:1428] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:29.924304 9545 tablet_service.cc:1467] Processing CreateTablet for tablet d64f7ae1e60b44e687b8b1e8065df3a5 (DEFAULT_TABLE table=test-table [id=b88b491207b74cbaa8dbef94b8cce753]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:39:29.924662 9545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d64f7ae1e60b44e687b8b1e8065df3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:39:29.925026 9749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:29.919646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:40907 (local address 127.0.0.1:43790)
0619 20:39:29.919814 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:29.921114 (+ 1300us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:29.921169 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:29.921337 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:29.921338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:29.921339 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:39:29.921382 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:29.921383 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.922496 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.922496 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:29.923538 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:29.923541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.924696 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.924697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:29.924839 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:29.924843 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:39:29.924901 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 20:39:29.924950 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":108,"threads_started":1}
I20260619 20:39:29.925439 9751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:29.919941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43790 (local address 127.25.62.131:40907)
0619 20:39:29.920227 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:29.920231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:29.921179 (+ 948us) server_negotiation.cc:407] Connection header received
0619 20:39:29.921238 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:29.921241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:29.921255 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:29.921342 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:39:29.921449 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.922402 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.923738 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:29.924466 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:29.924694 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:29.925313 (+ 619us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:29.925337 (+ 24us) server_negotiation.cc:299] Negotiation successful
0619 20:39:29.925375 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":91,"threads_started":1}
I20260619 20:39:29.926062 9677 tablet_service.cc:1467] Processing CreateTablet for tablet d64f7ae1e60b44e687b8b1e8065df3a5 (DEFAULT_TABLE table=test-table [id=b88b491207b74cbaa8dbef94b8cce753]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:39:29.926324 9677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d64f7ae1e60b44e687b8b1e8065df3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:39:29.927676 9753 tablet_bootstrap.cc:492] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69: Bootstrap starting.
I20260619 20:39:29.928511 9753 tablet_bootstrap.cc:654] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69: Neither blocks nor log segments found. Creating new log.
I20260619 20:39:29.929162 9753 log.cc:824] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69: Log is configured to *not* fsync() on all Append() calls
I20260619 20:39:29.929558 9755 tablet_bootstrap.cc:492] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5: Bootstrap starting.
I20260619 20:39:29.929947 9753 tablet_bootstrap.cc:492] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69: No bootstrap required, opened a new log
I20260619 20:39:29.930038 9753 ts_tablet_manager.cc:1397] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260619 20:39:29.930148 9755 tablet_bootstrap.cc:654] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5: Neither blocks nor log segments found. Creating new log.
I20260619 20:39:29.930617 9755 log.cc:824] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5: Log is configured to *not* fsync() on all Append() calls
I20260619 20:39:29.931324 9755 tablet_bootstrap.cc:492] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5: No bootstrap required, opened a new log
I20260619 20:39:29.931407 9755 ts_tablet_manager.cc:1397] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:39:29.931751 9753 raft_consensus.cc:348] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:29.931917 9753 raft_consensus.cc:374] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:39:29.931965 9753 raft_consensus.cc:729] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14d00dba0a6c4f8e9470116cefa36c69, State: Initialized, Role: FOLLOWER
I20260619 20:39:29.932089 9753 consensus_queue.cc:260] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:29.932400 9753 ts_tablet_manager.cc:1428] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 20:39:29.932417 9611 heartbeater.cc:503] Master 127.25.62.190:46749 was elected leader, sending a full tablet report...
I20260619 20:39:29.933419 9755 raft_consensus.cc:348] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:29.933597 9755 raft_consensus.cc:374] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:39:29.933635 9755 raft_consensus.cc:729] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 033c91b091ed4080b835cb06c5c976c5, State: Initialized, Role: FOLLOWER
I20260619 20:39:29.933730 9755 consensus_queue.cc:260] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:29.933928 9755 ts_tablet_manager.cc:1428] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:39:29.933974 9743 heartbeater.cc:503] Master 127.25.62.190:46749 was elected leader, sending a full tablet report...
W20260619 20:39:29.998790 9612 tablet.cc:2369] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:39:30.085937 9479 raft_consensus.cc:482] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:39:30.086126 9479 raft_consensus.cc:504] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:30.086476 9479 leader_election.cc:288] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14d00dba0a6c4f8e9470116cefa36c69 (127.25.62.130:44777), 033c91b091ed4080b835cb06c5c976c5 (127.25.62.131:40907)
I20260619 20:39:30.089241 9751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:30.086821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:49023 (local address 127.25.62.131:40907)
0619 20:39:30.086869 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:30.086871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:30.086879 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:39:30.086908 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:30.086909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:30.086918 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:30.086984 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:39:30.087125 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:30.087605 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:30.088511 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:30.088877 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:30.088962 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:30.088970 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:30.089139 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 20:39:30.089177 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:39:30.089241 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:30.086679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:40907 (local address 127.25.62.129:49023)
0619 20:39:30.086762 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:30.086775 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:30.086837 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:30.087002 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:30.087003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:30.087005 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:39:30.087063 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:30.087064 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:30.087685 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:30.087686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:30.088413 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:30.088415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:30.088960 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:30.088961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:30.089034 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:30.089071 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 20:39:30.089131 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 20:39:30.089170 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260619 20:39:30.089618 9697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d64f7ae1e60b44e687b8b1e8065df3a5" candidate_uuid: "eb7e685f461044bb9c20a8a8838efe61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "033c91b091ed4080b835cb06c5c976c5" is_pre_election: true
I20260619 20:39:30.089804 9697 raft_consensus.cc:2454] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb7e685f461044bb9c20a8a8838efe61 in term 0.
I20260619 20:39:30.090077 9750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:30.086749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:44915 (local address 127.25.62.130:44777)
0619 20:39:30.086798 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:30.086799 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:30.086930 (+ 131us) server_negotiation.cc:407] Connection header received
0619 20:39:30.087030 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:30.087032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:30.087046 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:30.087128 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:39:30.087556 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:30.088025 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:30.089121 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:30.089691 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:30.089787 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:30.089797 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:30.089964 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 20:39:30.090013 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 20:39:30.090076 9360 leader_election.cc:302] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 033c91b091ed4080b835cb06c5c976c5, eb7e685f461044bb9c20a8a8838efe61; no voters:
I20260619 20:39:30.090214 9759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:30.086679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:44777 (local address 127.25.62.129:44915)
0619 20:39:30.086844 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:30.086855 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:30.086942 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:30.087134 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:30.087134 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:30.087154 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:39:30.087472 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:30.087474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:30.088126 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:30.088127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:30.088993 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:30.088996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:30.089789 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:30.089790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:30.089852 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:30.089856 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:39:30.090014 (+ 158us) client_negotiation.cc:240] Negotiation successful
0619 20:39:30.090052 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":25,"thread_start_us":66,"threads_started":1}
I20260619 20:39:30.090247 9479 raft_consensus.cc:2781] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:39:30.090301 9479 raft_consensus.cc:482] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:39:30.090349 9479 raft_consensus.cc:3037] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:39:30.090533 9565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d64f7ae1e60b44e687b8b1e8065df3a5" candidate_uuid: "eb7e685f461044bb9c20a8a8838efe61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14d00dba0a6c4f8e9470116cefa36c69" is_pre_election: true
I20260619 20:39:30.090637 9565 raft_consensus.cc:2454] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb7e685f461044bb9c20a8a8838efe61 in term 0.
I20260619 20:39:30.091110 9479 raft_consensus.cc:504] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:30.091259 9479 leader_election.cc:288] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 1 election: Requested vote from peers 14d00dba0a6c4f8e9470116cefa36c69 (127.25.62.130:44777), 033c91b091ed4080b835cb06c5c976c5 (127.25.62.131:40907)
I20260619 20:39:30.091408 9565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d64f7ae1e60b44e687b8b1e8065df3a5" candidate_uuid: "eb7e685f461044bb9c20a8a8838efe61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14d00dba0a6c4f8e9470116cefa36c69"
I20260619 20:39:30.091431 9697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d64f7ae1e60b44e687b8b1e8065df3a5" candidate_uuid: "eb7e685f461044bb9c20a8a8838efe61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "033c91b091ed4080b835cb06c5c976c5"
I20260619 20:39:30.091480 9565 raft_consensus.cc:3037] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:39:30.091516 9697 raft_consensus.cc:3037] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:39:30.092231 9565 raft_consensus.cc:2454] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb7e685f461044bb9c20a8a8838efe61 in term 1.
I20260619 20:39:30.092240 9697 raft_consensus.cc:2454] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb7e685f461044bb9c20a8a8838efe61 in term 1.
I20260619 20:39:30.092401 9361 leader_election.cc:302] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14d00dba0a6c4f8e9470116cefa36c69, eb7e685f461044bb9c20a8a8838efe61; no voters:
I20260619 20:39:30.092515 9479 raft_consensus.cc:2781] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:39:30.092707 9479 raft_consensus.cc:686] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 1 LEADER]: Becoming Leader. State: Replica: eb7e685f461044bb9c20a8a8838efe61, State: Running, Role: LEADER
I20260619 20:39:30.092790 9479 consensus_queue.cc:237] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:30.093590 9285 catalog_manager.cc:5495] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb7e685f461044bb9c20a8a8838efe61 (127.25.62.129). New cstate: current_term: 1 leader_uuid: "eb7e685f461044bb9c20a8a8838efe61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } health_report { overall_health: HEALTHY } } }
I20260619 20:39:30.103914 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:30.101702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43424 (local address 127.25.62.129:39417)
0619 20:39:30.101759 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:30.101762 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:30.101771 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:39:30.101834 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:30.101834 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:30.101850 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:30.101928 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:39:30.102046 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:30.102510 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:30.103217 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:30.103402 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:30.103472 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:30.103630 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:39:30.103671 (+ 41us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 20:39:30.103817 (+ 146us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43424: token expired
Metrics: {"server-negotiator.queue_time_us":29}
W20260619 20:39:30.103973 9481 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43424: token expired
I20260619 20:39:30.111900 9476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:30.104464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39812 (local address 127.25.62.190:46749)
0619 20:39:30.104530 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:30.104532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:30.104541 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:39:30.104581 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:30.104583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:30.104595 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:30.104655 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:39:30.105418 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:30.106173 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:30.107276 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:30.107596 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:30.107677 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:30.109208 (+ 1531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:39:30.109210 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:39:30.109210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:39:30.109221 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:39:30.110649 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:30.111022 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:30.111023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:30.111023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:30.111108 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:30.111317 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:30.111318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:30.111318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:30.111453 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:39:30.111540 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:30.111656 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 20:39:30.111779 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:39:30.114974 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:30.112725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43434 (local address 127.25.62.129:39417)
0619 20:39:30.112775 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:30.112777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:30.112832 (+ 55us) server_negotiation.cc:407] Connection header received
0619 20:39:30.112893 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:30.112894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:30.112905 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:30.112964 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:39:30.113111 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:30.113548 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:30.114246 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:30.114438 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:30.114528 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:30.114665 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:39:30.114758 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:39:30.114817 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:30.114878 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 20:39:30.114896 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:39:30.116421 9697 raft_consensus.cc:1261] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5 [term 1 FOLLOWER]: Refusing update from remote peer eb7e685f461044bb9c20a8a8838efe61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:39:30.116421 9565 raft_consensus.cc:1261] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69 [term 1 FOLLOWER]: Refusing update from remote peer eb7e685f461044bb9c20a8a8838efe61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:39:30.116745 9479 consensus_queue.cc:1035] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:39:30.116880 9760 consensus_queue.cc:1035] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:39:30.120582 9763 mvcc.cc:205] Tried to move back new op lower bound from 7298668831194427392 to 7298668831100805120. Current Snapshot: MvccSnapshot[applied={T|T < 7298668831194427392 or (T in {7298668831194427392})}]
I20260619 20:39:30.120285 9764 mvcc.cc:205] Tried to move back new op lower bound from 7298668831194427392 to 7298668831100805120. Current Snapshot: MvccSnapshot[applied={T|T < 7298668831194427392}]
I20260619 20:39:30.122000 9285 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39812:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:39:30.122083 9285 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39812:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:39:30.122773 9285 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501: Sending DeleteTablet for 3 replicas of tablet d64f7ae1e60b44e687b8b1e8065df3a5
I20260619 20:39:30.123092 9545 tablet_service.cc:1514] Processing DeleteTablet for tablet d64f7ae1e60b44e687b8b1e8065df3a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:39:30 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:32968
I20260619 20:39:30.123106 9677 tablet_service.cc:1514] Processing DeleteTablet for tablet d64f7ae1e60b44e687b8b1e8065df3a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:39:30 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:43790
I20260619 20:39:30.123270 9405 tablet_service.cc:1514] Processing DeleteTablet for tablet d64f7ae1e60b44e687b8b1e8065df3a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:39:30 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:43416
I20260619 20:39:30.123313 9774 tablet_replica.cc:330] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5: stopping tablet replica
I20260619 20:39:30.123354 9482 tablet_replica.cc:330] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61: stopping tablet replica
I20260619 20:39:30.123413 9482 raft_consensus.cc:2229] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:39:30.123430 9774 raft_consensus.cc:2229] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:39:30.123497 9482 raft_consensus.cc:2258] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:39:30.123546 9774 raft_consensus.cc:2258] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:39:30.123719 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 9323
I20260619 20:39:30.123951 9774 ts_tablet_manager.cc:1905] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:39:30.125669 9774 ts_tablet_manager.cc:1918] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:39:30.125805 9774 log.cc:1196] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/d64f7ae1e60b44e687b8b1e8065df3a5
I20260619 20:39:30.126093 9773 tablet_replica.cc:330] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69: stopping tablet replica
I20260619 20:39:30.126152 9774 ts_tablet_manager.cc:1939] T d64f7ae1e60b44e687b8b1e8065df3a5 P 033c91b091ed4080b835cb06c5c976c5: Deleting consensus metadata
I20260619 20:39:30.126214 9773 raft_consensus.cc:2229] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:39:30.126341 9773 raft_consensus.cc:2258] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:39:30.126758 9773 ts_tablet_manager.cc:1905] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:39:30.126744 9270 catalog_manager.cc:4841] TS 033c91b091ed4080b835cb06c5c976c5 (127.25.62.131:40907): tablet d64f7ae1e60b44e687b8b1e8065df3a5 (table test-table [id=b88b491207b74cbaa8dbef94b8cce753]) successfully deleted
I20260619 20:39:30.128055 9773 ts_tablet_manager.cc:1918] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:39:30.128118 9773 log.cc:1196] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/d64f7ae1e60b44e687b8b1e8065df3a5
I20260619 20:39:30.128381 9773 ts_tablet_manager.cc:1939] T d64f7ae1e60b44e687b8b1e8065df3a5 P 14d00dba0a6c4f8e9470116cefa36c69: Deleting consensus metadata
I20260619 20:39:30.128784 9271 catalog_manager.cc:4841] TS 14d00dba0a6c4f8e9470116cefa36c69 (127.25.62.130:44777): tablet d64f7ae1e60b44e687b8b1e8065df3a5 (table test-table [id=b88b491207b74cbaa8dbef94b8cce753]) successfully deleted
W20260619 20:39:30.129648 9270 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.62.129:39417 (error 108)
I20260619 20:39:30.129930 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 9478
I20260619 20:39:30.130105 9749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:30.129958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:39417 (local address 127.0.0.1:43442)
0619 20:39:30.130019 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:30.130066 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.129:39417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 20:39:30.130648 9270 catalog_manager.cc:4584] TS eb7e685f461044bb9c20a8a8838efe61 (127.25.62.129:39417): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d64f7ae1e60b44e687b8b1e8065df3a5: Network error: Client connection negotiation failed: client connection to 127.25.62.129:39417: connect: Connection refused (error 111)
I20260619 20:39:30.135281 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 9614
I20260619 20:39:30.139605 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 9252
I20260619 20:39:50.144767 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:46749
--webserver_interface=127.25.62.190
--webserver_port=39817
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:46749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:39:50.223687 9776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:39:50.223928 9776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:39:50.223963 9776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:39:50.225107 9776 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 20:39:50.225170 9776 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 20:39:50.225191 9776 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 20:39:50.225211 9776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:39:50.225241 9776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:39:50.225267 9776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:39:50.225296 9776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:39:50.225322 9776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:39:50.226847 9776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:46749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:46749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=39817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:39:50.227113 9776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:39:50.227326 9776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:39:50.232031 9782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:39:50.232055 9784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:39:50.232055 9781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:39:50.232257 9776 server_base.cc:1034] running on GCE node
I20260619 20:39:50.232537 9776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:39:50.232872 9776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:39:50.234036 9776 hybrid_clock.cc:648] HybridClock initialized: now 1781901590234026 us; error 52 us; skew 500 ppm
Jun 19 20:39:50 dist-test-slave-0t40 krb5kdc[8702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901590, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:39:50.236501 9776 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:39:50.236922 9776 webserver.cc:460] Webserver started at http://127.25.62.190:39817/ using document root <none> and password file <none>
I20260619 20:39:50.237198 9776 fs_manager.cc:359] Metadata directory not provided
I20260619 20:39:50.237267 9776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:39:50.238570 9776 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:39:50.239321 9791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:50.239518 9776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:39:50.239622 9776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "9f6af0c094d2409e8a9f27895b305501"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "4231f42bd033cbe7a27d55b5f3d64fc7"
server_key_iv: "c9f269a8104bb4f75c68e0d2ce48af97"
server_key_version: "encryptionkey@0"
I20260619 20:39:50.239956 9776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:39:50.247311 9776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:39:50.249172 9776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:39:50.249315 9776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:39:50.253015 9776 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:46749
I20260619 20:39:50.253087 9843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:46749 every 8 connection(s)
I20260619 20:39:50.253456 9776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:39:50.254851 9844 sys_catalog.cc:263] Verifying existing consensus state
I20260619 20:39:50.255518 9844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501: Bootstrap starting.
I20260619 20:39:50.259452 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 9776
I20260619 20:39:50.259740 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:39417
--local_ip_for_outbound_sockets=127.25.62.129
--tserver_master_addrs=127.25.62.190:46749
--webserver_port=37511
--webserver_interface=127.25.62.129
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:39:50.264891 9844 log.cc:824] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501: Log is configured to *not* fsync() on all Append() calls
I20260619 20:39:50.267331 9844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:39:50.267666 9844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501: Bootstrap complete.
I20260619 20:39:50.269989 9844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6af0c094d2409e8a9f27895b305501" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46749 } }
I20260619 20:39:50.270246 9844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f6af0c094d2409e8a9f27895b305501, State: Initialized, Role: FOLLOWER
I20260619 20:39:50.270457 9844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6af0c094d2409e8a9f27895b305501" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46749 } }
I20260619 20:39:50.270562 9844 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:39:50.270619 9844 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:39:50.270673 9844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 2 FOLLOWER]: Advancing to term 3
I20260619 20:39:50.271679 9844 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6af0c094d2409e8a9f27895b305501" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46749 } }
I20260619 20:39:50.271806 9844 leader_election.cc:302] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f6af0c094d2409e8a9f27895b305501; no voters:
I20260619 20:39:50.272019 9844 leader_election.cc:288] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 20:39:50.272059 9848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 20:39:50.272248 9848 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [term 3 LEADER]: Becoming Leader. State: Replica: 9f6af0c094d2409e8a9f27895b305501, State: Running, Role: LEADER
I20260619 20:39:50.272303 9844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:39:50.272414 9848 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6af0c094d2409e8a9f27895b305501" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46749 } }
I20260619 20:39:50.274073 9849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "9f6af0c094d2409e8a9f27895b305501" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6af0c094d2409e8a9f27895b305501" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46749 } } }
I20260619 20:39:50.274160 9849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [sys.catalog]: This master's current role is: LEADER
I20260619 20:39:50.274296 9863 catalog_manager.cc:1259] Loaded cluster ID: c3518f7e531849e7a5f827d6bc0af70d
I20260619 20:39:50.274353 9863 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501: loading cluster ID for follower catalog manager: success
I20260619 20:39:50.274422 9850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f6af0c094d2409e8a9f27895b305501. Latest consensus state: current_term: 3 leader_uuid: "9f6af0c094d2409e8a9f27895b305501" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f6af0c094d2409e8a9f27895b305501" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46749 } } }
I20260619 20:39:50.274565 9850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501 [sys.catalog]: This master's current role is: LEADER
I20260619 20:39:50.275313 9863 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9f6af0c094d2409e8a9f27895b305501: acquiring CA information for follower catalog manager: success
F20260619 20:39:50.275651 9863 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa4f40199ad google::LogMessage::Fail() at ??:0
@ 0x7fa4f401c88a google::LogMessage::SendToLog() at ??:0
@ 0x7fa4f40194de google::LogMessage::Flush() at ??:0
@ 0x7fa4f401aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa4f6786104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fa4fbf766b9 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7fa4fbf76989 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7fa4fbf76e4e kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7fa4fbf798c2 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7fa4f5edc954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa4f6d406db start_thread at ??:0
@ 0x7fa4f2b1761f clone at ??:0
W20260619 20:39:50.356609 9846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:39:50.356840 9846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:39:50.356881 9846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:39:50.358073 9846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:39:50.358145 9846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:39:50.358193 9846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:39:50.359658 9846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:39417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=37511
--tserver_master_addrs=127.25.62.190:46749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:39:50.359930 9846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:39:50.360172 9846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:39:50.365267 9883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:39:50.365388 9885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:39:50.365255 9882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:39:50.365398 9846 server_base.cc:1034] running on GCE node
I20260619 20:39:50.365805 9846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:39:50.366110 9846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:39:50.367302 9846 hybrid_clock.cc:648] HybridClock initialized: now 1781901590367290 us; error 40 us; skew 500 ppm
Jun 19 20:39:50 dist-test-slave-0t40 krb5kdc[8702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901590, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:39:50.370213 9846 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:39:50.370673 9846 webserver.cc:460] Webserver started at http://127.25.62.129:37511/ using document root <none> and password file <none>
I20260619 20:39:50.370909 9846 fs_manager.cc:359] Metadata directory not provided
I20260619 20:39:50.370978 9846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:39:50.372352 9846 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:39:50.373178 9892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:50.374251 9846 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 20:39:50.374351 9846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "eb7e685f461044bb9c20a8a8838efe61"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "27cd64a84fac91b22f72739059f2c960"
server_key_iv: "584a70992d78548f10a5524a0c7203b7"
server_key_version: "encryptionkey@0"
I20260619 20:39:50.374722 9846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:39:50.384130 9846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:39:50.386358 9846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:39:50.386514 9846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:39:50.386729 9846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:39:50.387147 9899 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 20:39:50.389110 9846 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 20:39:50.389186 9846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 20:39:50.389230 9846 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 20:39:50.389830 9846 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 20:39:50.389885 9846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 20:39:50.389964 9899 tablet_bootstrap.cc:492] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61: Bootstrap starting.
I20260619 20:39:50.396998 9846 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:39417
I20260619 20:39:50.397075 10006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:39417 every 8 connection(s)
I20260619 20:39:50.397457 9846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:39:50.401119 9899 log.cc:824] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61: Log is configured to *not* fsync() on all Append() calls
I20260619 20:39:50.403788 9899 tablet_bootstrap.cc:492] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 20:39:50.404124 9899 tablet_bootstrap.cc:492] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61: Bootstrap complete.
I20260619 20:39:50.405362 9899 ts_tablet_manager.cc:1397] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.011s
I20260619 20:39:50.405471 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 9846
I20260619 20:39:50.405763 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:44777
--local_ip_for_outbound_sockets=127.25.62.130
--tserver_master_addrs=127.25.62.190:46749
--webserver_port=33683
--webserver_interface=127.25.62.130
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:39:50.406806 9899 raft_consensus.cc:348] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:50.407140 9899 raft_consensus.cc:729] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb7e685f461044bb9c20a8a8838efe61, State: Initialized, Role: FOLLOWER
I20260619 20:39:50.407308 9899 consensus_queue.cc:260] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:50.407578 9899 ts_tablet_manager.cc:1428] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260619 20:39:50.475188 10009 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:39:50.398162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:35685)
0619 20:39:50.398332 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:50.398373 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:50.398804 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:50.474978 (+ 76174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
W20260619 20:39:50.475509 10007 heartbeater.cc:644] Failed to heartbeat to 127.25.62.190:46749 (0 consecutive failures): Network error: Failed to ping master at 127.25.62.190:46749: Client connection negotiation failed: client connection to 127.25.62.190:46749: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 20:39:50.475898 10009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:50.475772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:47035)
0619 20:39:50.475826 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:50.475873 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 20:39:50.476243 10009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:50.476141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:32903)
0619 20:39:50.476192 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:50.476221 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 20:39:50.476385 10007 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 20:39:50.495744 10011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:39:50.495950 10011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:39:50.495973 10011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:39:50.497153 10011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:39:50.497193 10011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:39:50.497242 10011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:39:50.498639 10011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:44777
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=33683
--tserver_master_addrs=127.25.62.190:46749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:39:50.498847 10011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:39:50.499058 10011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:39:50.503945 10017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:39:50.503989 10018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:39:50.503935 10020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:39:50.504101 10011 server_base.cc:1034] running on GCE node
I20260619 20:39:50.504326 10011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:39:50.504506 10011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:39:50.505650 10011 hybrid_clock.cc:648] HybridClock initialized: now 1781901590505634 us; error 29 us; skew 500 ppm
Jun 19 20:39:50 dist-test-slave-0t40 krb5kdc[8702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901590, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:39:50.508186 10011 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:39:50.508646 10011 webserver.cc:460] Webserver started at http://127.25.62.130:33683/ using document root <none> and password file <none>
I20260619 20:39:50.508878 10011 fs_manager.cc:359] Metadata directory not provided
I20260619 20:39:50.508960 10011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:39:50.510293 10011 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:39:50.510922 10027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:50.511158 10011 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:39:50.511250 10011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "14d00dba0a6c4f8e9470116cefa36c69"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "0f7dde673f0c82b0b72f05ce7dbd33b6"
server_key_iv: "6060b90292c99ab92382dbac44e2c5f6"
server_key_version: "encryptionkey@0"
I20260619 20:39:50.511567 10011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:39:50.522579 10011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:39:50.524343 10011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:39:50.524448 10011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:39:50.524612 10011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:39:50.524899 10011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:39:50.524974 10011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:50.525022 10011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:39:50.525111 10011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:50.531033 10011 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:44777
I20260619 20:39:50.531113 10140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:44777 every 8 connection(s)
I20260619 20:39:50.531440 10011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:39:50.532428 10143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:50.532082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:58043)
0619 20:39:50.532224 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:50.532293 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
W20260619 20:39:50.532857 10141 heartbeater.cc:644] Failed to heartbeat to 127.25.62.190:46749 (0 consecutive failures): Network error: Failed to ping master at 127.25.62.190:46749: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
I20260619 20:39:50.533164 10143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:50.533082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:34333)
0619 20:39:50.533114 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:50.533148 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260619 20:39:50.533435 10143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:50.533387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:41767)
0619 20:39:50.533406 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:50.533423 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260619 20:39:50.533552 10141 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 20:39:50.541497 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 10011
I20260619 20:39:50.541764 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:40907
--local_ip_for_outbound_sockets=127.25.62.131
--tserver_master_addrs=127.25.62.190:46749
--webserver_port=44565
--webserver_interface=127.25.62.131
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:39:50.620445 10144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:39:50.620664 10144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:39:50.620700 10144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:39:50.621832 10144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:39:50.621896 10144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:39:50.621939 10144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:39:50.623358 10144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:40907
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=44565
--tserver_master_addrs=127.25.62.190:46749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:39:50.623615 10144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:39:50.623845 10144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:39:50.629144 10149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:39:50.629141 10150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:39:50.629172 10144 server_base.cc:1034] running on GCE node
W20260619 20:39:50.629210 10152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:39:50.629585 10144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:39:50.629837 10144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:39:50.630995 10144 hybrid_clock.cc:648] HybridClock initialized: now 1781901590630974 us; error 39 us; skew 500 ppm
Jun 19 20:39:50 dist-test-slave-0t40 krb5kdc[8702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901590, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:39:50.633535 10144 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:39:50.633946 10144 webserver.cc:460] Webserver started at http://127.25.62.131:44565/ using document root <none> and password file <none>
I20260619 20:39:50.634186 10144 fs_manager.cc:359] Metadata directory not provided
I20260619 20:39:50.634254 10144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:39:50.635542 10144 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:39:50.636274 10159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:50.636474 10144 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:39:50.636571 10144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "033c91b091ed4080b835cb06c5c976c5"
format_stamp: "Formatted at 2026-06-19 20:39:08 on dist-test-slave-0t40"
server_key: "b93e39b2351f193b70454e6aec133fc0"
server_key_iv: "4e2924017a5500ffdfd132f0b04902ba"
server_key_version: "encryptionkey@0"
I20260619 20:39:50.636909 10144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:39:50.650399 10144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:39:50.652215 10144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:39:50.652375 10144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:39:50.652591 10144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:39:50.652879 10144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:39:50.652931 10144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:50.652972 10144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:39:50.653000 10144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:39:50.659190 10144 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:40907
I20260619 20:39:50.659229 10272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:40907 every 8 connection(s)
I20260619 20:39:50.659574 10144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:39:50.660852 10275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:50.660317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:60935)
0619 20:39:50.660515 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:50.660616 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
W20260619 20:39:50.661420 10273 heartbeater.cc:644] Failed to heartbeat to 127.25.62.190:46749 (0 consecutive failures): Network error: Failed to ping master at 127.25.62.190:46749: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
I20260619 20:39:50.661741 10275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:50.661631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:43959)
0619 20:39:50.661661 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:50.661720 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260619 20:39:50.662099 10275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:50.661992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:59777)
0619 20:39:50.662036 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:50.662055 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 20:39:50.662235 10273 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 20:39:50.666318 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 10144
I20260619 20:39:51.477063 10277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:51.476822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:39575)
0619 20:39:51.476946 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:51.476981 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":58,"threads_started":1}
I20260619 20:39:51.534051 10278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:51.533860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:34227)
0619 20:39:51.533974 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:51.534008 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":52,"threads_started":1}
I20260619 20:39:51.662773 10279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:51.662537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:57915)
0619 20:39:51.662654 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:51.662704 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":73,"threads_started":1}
I20260619 20:39:51.714879 10280 raft_consensus.cc:482] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:39:51.715044 10280 raft_consensus.cc:504] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:51.715436 10280 leader_election.cc:288] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 14d00dba0a6c4f8e9470116cefa36c69 (127.25.62.130:44777), 033c91b091ed4080b835cb06c5c976c5 (127.25.62.131:40907)
Jun 19 20:39:51 dist-test-slave-0t40 krb5kdc[8702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901590, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.131@KRBTEST.COM
Jun 19 20:39:51 dist-test-slave-0t40 krb5kdc[8702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901590, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.130@KRBTEST.COM
I20260619 20:39:51.731419 10283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:51.715703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:59345 (local address 127.25.62.131:40907)
0619 20:39:51.715852 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:51.715859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:51.715885 (+ 26us) server_negotiation.cc:407] Connection header received
0619 20:39:51.716420 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:51.716423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:51.716484 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:51.716635 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:39:51.717703 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:51.719220 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:51.720791 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:51.721455 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:51.721571 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:51.727689 (+ 6118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:39:51.727694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:39:51.727694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:39:51.727704 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:39:51.729598 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:51.730085 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:51.730089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:51.730089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:51.730200 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:51.730486 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:51.730487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:51.730487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:51.730773 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:39:51.730847 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:51.731178 (+ 331us) server_negotiation.cc:299] Negotiation successful
0619 20:39:51.731306 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260619 20:39:51.731419 10277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:51.715548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:44777 (local address 127.25.62.129:44883)
0619 20:39:51.715596 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:51.715606 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:51.715660 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:51.716707 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:51.716710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:51.717269 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:39:51.717557 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:51.717565 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:51.719360 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:51.719361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:51.720692 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:51.720695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:51.721568 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:51.721568 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:51.721730 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:51.725892 (+ 4162us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:39:51.725905 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:39:51.727987 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:39:51.729986 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:39:51.729987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:39:51.729987 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:39:51.730209 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:39:51.730486 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:39:51.730486 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:39:51.730487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:39:51.730551 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:39:51.730964 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:39:51.730965 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:39:51.731008 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 20:39:51.731128 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 20:39:51.731325 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":108}
I20260619 20:39:51.731451 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:51.715635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:44883 (local address 127.25.62.130:44777)
0619 20:39:51.715780 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 20:39:51.715794 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 20:39:51.715829 (+ 35us) server_negotiation.cc:407] Connection header received
0619 20:39:51.716420 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:39:51.716423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:39:51.716482 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:39:51.716635 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:39:51.717703 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:51.719220 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:51.720791 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:39:51.721456 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:39:51.721571 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:51.728110 (+ 6539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:39:51.728114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:39:51.728114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:39:51.728124 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:39:51.729893 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:51.730296 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:51.730298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:51.730299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:51.730394 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:39:51.730616 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:39:51.730616 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:39:51.730617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:39:51.730887 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:39:51.730967 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 20:39:51.731179 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 20:39:51.731315 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":55,"threads_started":1}
I20260619 20:39:51.731531 10281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:51.715592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:40907 (local address 127.25.62.129:59345)
0619 20:39:51.715704 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:51.715714 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:39:51.715764 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:39:51.716707 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:39:51.716710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:39:51.717273 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:39:51.717558 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:51.717565 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:51.719343 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:51.719344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:51.720692 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:39:51.720695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:39:51.721566 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:39:51.721567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:39:51.721731 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:39:51.725915 (+ 4184us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:39:51.725920 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:39:51.727562 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:39:51.729683 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:39:51.729691 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:39:51.729693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:39:51.730005 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:39:51.730288 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:39:51.730289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:39:51.730289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:39:51.730420 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:39:51.730857 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:39:51.730863 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:39:51.731008 (+ 145us) client_negotiation.cc:769] Sending connection context
0619 20:39:51.731127 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 20:39:51.731402 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":199,"thread_start_us":58,"threads_started":1}
W20260619 20:39:51.732481 9894 leader_election.cc:341] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 033c91b091ed4080b835cb06c5c976c5 (127.25.62.131:40907): Not found: Tablet not found: d64f7ae1e60b44e687b8b1e8065df3a5
W20260619 20:39:51.732613 9895 leader_election.cc:341] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 14d00dba0a6c4f8e9470116cefa36c69 (127.25.62.130:44777): Not found: Tablet not found: d64f7ae1e60b44e687b8b1e8065df3a5
I20260619 20:39:51.732651 9895 leader_election.cc:302] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb7e685f461044bb9c20a8a8838efe61; no voters: 033c91b091ed4080b835cb06c5c976c5, 14d00dba0a6c4f8e9470116cefa36c69
I20260619 20:39:51.732769 10280 raft_consensus.cc:2726] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:39:52.477764 10284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:52.477558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:39933)
0619 20:39:52.477678 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:52.477717 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260619 20:39:52.534626 10285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:52.534437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:39089)
0619 20:39:52.534559 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:52.534591 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260619 20:39:52.663511 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:52.663230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:44437)
0619 20:39:52.663421 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:52.663464 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":109,"threads_started":1}
I20260619 20:39:53.272599 10287 raft_consensus.cc:482] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:39:53.272691 10287 raft_consensus.cc:504] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:53.272862 10287 leader_election.cc:288] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 14d00dba0a6c4f8e9470116cefa36c69 (127.25.62.130:44777), 033c91b091ed4080b835cb06c5c976c5 (127.25.62.131:40907)
W20260619 20:39:53.273293 9895 leader_election.cc:341] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 14d00dba0a6c4f8e9470116cefa36c69 (127.25.62.130:44777): Not found: Tablet not found: d64f7ae1e60b44e687b8b1e8065df3a5
W20260619 20:39:53.273355 9894 leader_election.cc:341] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 033c91b091ed4080b835cb06c5c976c5 (127.25.62.131:40907): Not found: Tablet not found: d64f7ae1e60b44e687b8b1e8065df3a5
I20260619 20:39:53.273382 9895 leader_election.cc:302] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb7e685f461044bb9c20a8a8838efe61; no voters: 033c91b091ed4080b835cb06c5c976c5, 14d00dba0a6c4f8e9470116cefa36c69
I20260619 20:39:53.273450 10287 raft_consensus.cc:2726] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:39:53.478570 10288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:53.478310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:47195)
0619 20:39:53.478483 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:53.478524 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":105,"threads_started":1}
I20260619 20:39:53.535290 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:53.535085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:37609)
0619 20:39:53.535214 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:53.535252 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260619 20:39:53.664222 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:53.664024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:49847)
0619 20:39:53.664143 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:53.664177 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":81,"threads_started":1}
I20260619 20:39:54.479288 10291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:54.479042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:47899)
0619 20:39:54.479195 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:54.479233 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
I20260619 20:39:54.536139 10292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:54.535895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:38417)
0619 20:39:54.536052 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:54.536095 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":120,"threads_started":1}
I20260619 20:39:54.664917 10293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:54.664689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:37891)
0619 20:39:54.664826 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:54.664869 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":61,"threads_started":1}
I20260619 20:39:55.480014 10294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:55.479805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:48427)
0619 20:39:55.479930 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:55.479969 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260619 20:39:55.536844 10295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:55.536625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:37277)
0619 20:39:55.536759 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:55.536799 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":56,"threads_started":1}
I20260619 20:39:55.665557 10296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:55.665358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:48785)
0619 20:39:55.665481 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:55.665516 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260619 20:39:56.372439 10297 raft_consensus.cc:482] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:39:56.372545 10297 raft_consensus.cc:504] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:39:56.372723 10297 leader_election.cc:288] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 14d00dba0a6c4f8e9470116cefa36c69 (127.25.62.130:44777), 033c91b091ed4080b835cb06c5c976c5 (127.25.62.131:40907)
W20260619 20:39:56.373183 9895 leader_election.cc:341] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 14d00dba0a6c4f8e9470116cefa36c69 (127.25.62.130:44777): Not found: Tablet not found: d64f7ae1e60b44e687b8b1e8065df3a5
W20260619 20:39:56.373301 9894 leader_election.cc:341] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 033c91b091ed4080b835cb06c5c976c5 (127.25.62.131:40907): Not found: Tablet not found: d64f7ae1e60b44e687b8b1e8065df3a5
I20260619 20:39:56.373339 9894 leader_election.cc:302] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb7e685f461044bb9c20a8a8838efe61; no voters: 033c91b091ed4080b835cb06c5c976c5, 14d00dba0a6c4f8e9470116cefa36c69
I20260619 20:39:56.373442 10297 raft_consensus.cc:2726] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:39:56.480657 10298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:56.480461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:38893)
0619 20:39:56.480575 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:56.480612 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260619 20:39:56.537487 10299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:56.537296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:39301)
0619 20:39:56.537414 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:56.537450 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260619 20:39:56.666178 10300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:56.665983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:36351)
0619 20:39:56.666092 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:56.666130 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":66,"threads_started":1}
I20260619 20:39:57.481352 10301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:57.481151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:55013)
0619 20:39:57.481269 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:57.481307 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260619 20:39:57.538125 10302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:57.537910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:45673)
0619 20:39:57.538049 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:57.538087 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":62,"threads_started":1}
I20260619 20:39:57.666767 10303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:57.666561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:33757)
0619 20:39:57.666687 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:57.666724 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":73,"threads_started":1}
I20260619 20:39:58.482024 10304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:58.481789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:40011)
0619 20:39:58.481933 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:58.481978 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260619 20:39:58.538755 10305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:58.538565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:58433)
0619 20:39:58.538680 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:58.538714 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
I20260619 20:39:58.667380 10306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:58.667173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:32975)
0619 20:39:58.667312 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:58.667342 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260619 20:39:59.482764 10307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:59.482521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:55589)
0619 20:39:59.482666 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:59.482711 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":61,"threads_started":1}
I20260619 20:39:59.539465 10308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:59.539214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:33345)
0619 20:39:59.539338 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:59.539418 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":57,"threads_started":1}
I20260619 20:39:59.668038 10309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:39:59.667828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:45955)
0619 20:39:59.667963 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 20:39:59.667996 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260619 20:40:00.483516 10310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:00.483309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:50759)
0619 20:40:00.483429 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:00.483468 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260619 20:40:00.540266 10311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:00.539924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:38439)
0619 20:40:00.540172 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:00.540221 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":139,"threads_started":1}
I20260619 20:40:00.668803 10312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:00.668547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:36303)
0619 20:40:00.668699 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:00.668746 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
I20260619 20:40:01.377426 10313 raft_consensus.cc:482] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:40:01.377516 10313 raft_consensus.cc:504] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14d00dba0a6c4f8e9470116cefa36c69" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44777 } } peers { permanent_uuid: "033c91b091ed4080b835cb06c5c976c5" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 40907 } } peers { permanent_uuid: "eb7e685f461044bb9c20a8a8838efe61" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 39417 } }
I20260619 20:40:01.377707 10313 leader_election.cc:288] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 14d00dba0a6c4f8e9470116cefa36c69 (127.25.62.130:44777), 033c91b091ed4080b835cb06c5c976c5 (127.25.62.131:40907)
W20260619 20:40:01.378062 9894 leader_election.cc:341] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 033c91b091ed4080b835cb06c5c976c5 (127.25.62.131:40907): Not found: Tablet not found: d64f7ae1e60b44e687b8b1e8065df3a5
W20260619 20:40:01.378154 9895 leader_election.cc:341] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 14d00dba0a6c4f8e9470116cefa36c69 (127.25.62.130:44777): Not found: Tablet not found: d64f7ae1e60b44e687b8b1e8065df3a5
I20260619 20:40:01.378242 9895 leader_election.cc:302] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb7e685f461044bb9c20a8a8838efe61; no voters: 033c91b091ed4080b835cb06c5c976c5, 14d00dba0a6c4f8e9470116cefa36c69
I20260619 20:40:01.378310 10313 raft_consensus.cc:2726] T d64f7ae1e60b44e687b8b1e8065df3a5 P eb7e685f461044bb9c20a8a8838efe61 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 20:40:01.484253 10314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:01.484017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:48517)
0619 20:40:01.484162 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:01.484204 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":61,"threads_started":1}
I20260619 20:40:01.541121 10315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:01.540791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:45809)
0619 20:40:01.540953 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:01.541004 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260619 20:40:01.669505 10316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:01.669252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:37153)
0619 20:40:01.669399 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:01.669435 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":60,"threads_started":1}
I20260619 20:40:02.485140 10317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:02.484828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:48567)
0619 20:40:02.484990 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:02.485036 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":66,"threads_started":1}
I20260619 20:40:02.541826 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:02.541602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:34161)
0619 20:40:02.541731 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:02.541781 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260619 20:40:02.670313 10319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:02.670054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:36977)
0619 20:40:02.670189 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:02.670246 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260619 20:40:03.485826 10320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:03.485609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:38897)
0619 20:40:03.485741 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:03.485779 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260619 20:40:03.542547 10321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:03.542342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:55657)
0619 20:40:03.542463 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:03.542502 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":58,"threads_started":1}
I20260619 20:40:03.670879 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:03.670691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:38767)
0619 20:40:03.670805 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:03.670837 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":52,"threads_started":1}
I20260619 20:40:04.486629 10323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:04.486365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:59265)
0619 20:40:04.486499 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:04.486540 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
I20260619 20:40:04.543249 10324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:04.543032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:32951)
0619 20:40:04.543168 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:04.543205 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":60,"threads_started":1}
I20260619 20:40:04.671547 10325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:04.671320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:33599)
0619 20:40:04.671464 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:04.671503 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":61,"threads_started":1}
I20260619 20:40:05.487484 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:05.487258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.129:36065)
0619 20:40:05.487396 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:05.487435 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":71,"threads_started":1}
I20260619 20:40:05.544144 10327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:05.543894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.130:47667)
0619 20:40:05.544065 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:05.544099 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":113,"threads_started":1}
I20260619 20:40:05.669611 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 9846
I20260619 20:40:05.672293 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:05.672077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46749 (local address 127.25.62.131:51729)
0619 20:40:05.672192 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:05.672232 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.190:46749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":52,"threads_started":1}
I20260619 20:40:05.674633 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 10011
I20260619 20:40:05.678685 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 10144
2026-06-19T20:40:05Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58496 ms)
[----------] 30 tests from SecurityITest (227459 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:05 dist-test-slave-0t40 krb5kdc[10331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:05 dist-test-slave-0t40 krb5kdc[10331](info): set up 2 sockets
Jun 19 20:40:05 dist-test-slave-0t40 krb5kdc[10331](info): commencing operation
krb5kdc: starting...
W20260619 20:40:06.741528 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:06 dist-test-slave-0t40 krb5kdc[10331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:40:06.859337 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:45871
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:39609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:45871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:06.937469 10344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:06.937703 10344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:06.937740 10344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:06.938889 10344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:06.938964 10344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:06.938992 10344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:06.939024 10344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:06.939047 10344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:06.940526 10344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:39609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:45871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:45871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:06.940793 10344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:06.941021 10344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:06.946148 10349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:06.946199 10352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:06.946156 10350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:06.946187 10344 server_base.cc:1034] running on GCE node
I20260619 20:40:06.946619 10344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:06.946916 10344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:06.948082 10344 hybrid_clock.cc:648] HybridClock initialized: now 1781901606948068 us; error 27 us; skew 500 ppm
Jun 19 20:40:06 dist-test-slave-0t40 krb5kdc[10331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901606, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:06.950762 10344 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:40:06.951221 10344 webserver.cc:460] Webserver started at http://127.25.62.190:42263/ using document root <none> and password file <none>
I20260619 20:40:06.951464 10344 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:06.951532 10344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:06.951678 10344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:06.952737 10344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "ff2fa5271dec4c9e8ee52f657172324f"
format_stamp: "Formatted at 2026-06-19 20:40:06 on dist-test-slave-0t40"
server_key: "00afc80f27c7ea35200c9ad010d5d9b7"
server_key_iv: "a376064aaba156bb24b8e028db7bd88b"
server_key_version: "encryptionkey@0"
I20260619 20:40:06.953166 10344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "ff2fa5271dec4c9e8ee52f657172324f"
format_stamp: "Formatted at 2026-06-19 20:40:06 on dist-test-slave-0t40"
server_key: "00afc80f27c7ea35200c9ad010d5d9b7"
server_key_iv: "a376064aaba156bb24b8e028db7bd88b"
server_key_version: "encryptionkey@0"
I20260619 20:40:06.954555 10344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:06.955556 10359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:06.955804 10344 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:06.955912 10344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "ff2fa5271dec4c9e8ee52f657172324f"
format_stamp: "Formatted at 2026-06-19 20:40:06 on dist-test-slave-0t40"
server_key: "00afc80f27c7ea35200c9ad010d5d9b7"
server_key_iv: "a376064aaba156bb24b8e028db7bd88b"
server_key_version: "encryptionkey@0"
I20260619 20:40:06.956008 10344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:06.984143 10344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:06.986522 10344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:06.986688 10344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:06.990833 10344 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:45871
I20260619 20:40:06.990896 10411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:45871 every 8 connection(s)
I20260619 20:40:06.991245 10344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:06.992103 10412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:06.993856 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 10344
I20260619 20:40:06.993961 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:06.994140 25850 external_mini_cluster.cc:1428] Setting key 2a85e2250dedc01f0a26b0fa3afff39d
I20260619 20:40:06.994727 10412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f: Bootstrap starting.
I20260619 20:40:06.995383 10412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:06.995755 10412 log.cc:824] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:06.996495 10412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f: No bootstrap required, opened a new log
I20260619 20:40:06.998803 10412 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2fa5271dec4c9e8ee52f657172324f" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 45871 } }
I20260619 20:40:06.998976 10412 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:06.999063 10412 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2fa5271dec4c9e8ee52f657172324f, State: Initialized, Role: FOLLOWER
I20260619 20:40:06.999214 10412 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2fa5271dec4c9e8ee52f657172324f" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 45871 } }
I20260619 20:40:06.999311 10412 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:06.999361 10412 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:06.999413 10412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:06.999975 10412 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2fa5271dec4c9e8ee52f657172324f" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 45871 } }
I20260619 20:40:07.000099 10412 leader_election.cc:302] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff2fa5271dec4c9e8ee52f657172324f; no voters:
I20260619 20:40:07.000299 10412 leader_election.cc:288] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:07.000324 10417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:07.000440 10417 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f [term 1 LEADER]: Becoming Leader. State: Replica: ff2fa5271dec4c9e8ee52f657172324f, State: Running, Role: LEADER
I20260619 20:40:07.000567 10412 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:07.000582 10417 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2fa5271dec4c9e8ee52f657172324f" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 45871 } }
I20260619 20:40:07.000919 10418 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff2fa5271dec4c9e8ee52f657172324f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2fa5271dec4c9e8ee52f657172324f" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 45871 } } }
Jun 19 20:40:06 dist-test-slave-0t40 krb5kdc[10331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901606, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:40:07.001170 10418 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:07.000906 10417 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff2fa5271dec4c9e8ee52f657172324f. Latest consensus state: current_term: 1 leader_uuid: "ff2fa5271dec4c9e8ee52f657172324f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff2fa5271dec4c9e8ee52f657172324f" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 45871 } } }
I20260619 20:40:07.001672 10417 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:07.001822 10426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:07.002477 10426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:07.004248 10426 catalog_manager.cc:1347] Generated new cluster ID: c3b19962940d4d3ba3d270440f51f716
I20260619 20:40:07.004304 10426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:07.018246 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:06.994685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35546 (local address 127.25.62.190:45871)
0619 20:40:06.994861 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:06.994868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:06.994893 (+ 25us) server_negotiation.cc:407] Connection header received
0619 20:40:06.995310 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:06.995317 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:06.995359 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:06.995514 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:06.996342 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:06.997330 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:06.998438 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:06.998880 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:06.998959 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.001664 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:07.001668 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:07.001668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:07.001680 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:07.003509 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:07.003916 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:07.003919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:07.003920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:07.004013 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:07.017117 (+ 13104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:07.017119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:07.017119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:07.017455 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:07.017708 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:07.017793 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 20:40:07.017967 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260619 20:40:07.026789 10426 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:07.027464 10426 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:07.032840 10426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f: Generated new TSK 0
I20260619 20:40:07.033071 10426 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
I20260619 20:40:07.063721 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:45871
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:39609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:07.146291 10438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:07.146481 10438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:07.146504 10438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:07.147562 10438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:07.147598 10438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:07.147624 10438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:40:07.148983 10438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:39609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:45871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:07.149230 10438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:07.149406 10438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:07.154232 10444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:07.154261 10446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:07.154353 10438 server_base.cc:1034] running on GCE node
W20260619 20:40:07.154223 10443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:07.154733 10438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:07.154992 10438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:07.156150 10438 hybrid_clock.cc:648] HybridClock initialized: now 1781901607156149 us; error 35 us; skew 500 ppm
Jun 19 20:40:07 dist-test-slave-0t40 krb5kdc[10331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901607, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:07.158602 10438 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:40:07.159053 10438 webserver.cc:460] Webserver started at http://127.25.62.129:33219/ using document root <none> and password file <none>
I20260619 20:40:07.159269 10438 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:07.159339 10438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:07.159482 10438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:07.160524 10438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "8970135a18a6419eb3519b6526172148"
format_stamp: "Formatted at 2026-06-19 20:40:07 on dist-test-slave-0t40"
server_key: "ca262afcf3ab7b883e1b838b5e976cff"
server_key_iv: "c60ed7bf3c894213dd77f6f2e87362dc"
server_key_version: "encryptionkey@0"
I20260619 20:40:07.160878 10438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "8970135a18a6419eb3519b6526172148"
format_stamp: "Formatted at 2026-06-19 20:40:07 on dist-test-slave-0t40"
server_key: "ca262afcf3ab7b883e1b838b5e976cff"
server_key_iv: "c60ed7bf3c894213dd77f6f2e87362dc"
server_key_version: "encryptionkey@0"
I20260619 20:40:07.162158 10438 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:40:07.162833 10453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:07.163003 10438 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:40:07.163095 10438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "8970135a18a6419eb3519b6526172148"
format_stamp: "Formatted at 2026-06-19 20:40:07 on dist-test-slave-0t40"
server_key: "ca262afcf3ab7b883e1b838b5e976cff"
server_key_iv: "c60ed7bf3c894213dd77f6f2e87362dc"
server_key_version: "encryptionkey@0"
I20260619 20:40:07.163172 10438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:07.174952 10438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:07.177237 10438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:07.177397 10438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:07.177616 10438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:40:07.177906 10438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:40:07.177955 10438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:07.177994 10438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:40:07.178025 10438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:07.183634 10438 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:33247
I20260619 20:40:07.183674 10566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:33247 every 8 connection(s)
I20260619 20:40:07.184000 10438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:40:07.188639 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 10438
I20260619 20:40:07.188752 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:40:07.188966 25850 external_mini_cluster.cc:1428] Setting key e00c00d6d98151a21431a9a174bd46d5
Jun 19 20:40:07 dist-test-slave-0t40 krb5kdc[10331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901607, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:40:07.193252 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.184678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:36611 (local address 127.25.62.190:45871)
0619 20:40:07.184760 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:07.184762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:07.184917 (+ 155us) server_negotiation.cc:407] Connection header received
0619 20:40:07.185378 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:07.185379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:07.185388 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:07.185439 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:07.186302 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.186807 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.187740 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.187935 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.187986 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.190098 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:07.190100 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:07.190100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:07.190108 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:07.191636 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:07.192154 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:07.192155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:07.192156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:07.192217 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:07.192516 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:07.192516 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:07.192517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:07.192628 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:07.192680 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:07.193029 (+ 349us) server_negotiation.cc:299] Negotiation successful
0619 20:40:07.193183 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 20:40:07.193725 10569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.184631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:45871 (local address 127.25.62.129:36611)
0619 20:40:07.184803 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:07.184838 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:07.185300 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:07.185488 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:07.185491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:07.186029 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:07.186211 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:07.186215 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.186887 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.186888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:07.187671 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:07.187673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.188007 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.188007 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:07.188103 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.188521 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:07.188528 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:07.190001 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:07.191720 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:07.191726 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:07.191728 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:07.192061 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:07.192294 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:07.192295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:07.192296 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:07.192452 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:07.192694 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:07.192700 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:07.192864 (+ 164us) client_negotiation.cc:769] Sending connection context
0619 20:40:07.192983 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 20:40:07.193197 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260619 20:40:07.194245 10567 heartbeater.cc:347] Connected to a master server at 127.25.62.190:45871
I20260619 20:40:07.194356 10567 heartbeater.cc:464] Registering TS with master...
I20260619 20:40:07.194591 10567 heartbeater.cc:511] Master 127.25.62.190:45871 requested a full tablet report, sending...
I20260619 20:40:07.195108 10376 ts_manager.cc:194] Registered new tserver with Master: 8970135a18a6419eb3519b6526172148 (127.25.62.129:33247)
I20260619 20:40:07.195749 10376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:36611
WARNING: no policy specified for kudu/127.25.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
I20260619 20:40:07.215637 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:45871
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:39609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:07.294955 10572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:07.295162 10572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:07.295186 10572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:07.296271 10572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:07.296306 10572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:07.296332 10572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:40:07.297699 10572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:39609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:45871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:07.297874 10572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:07.298059 10572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:07.303900 10580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:07.303932 10572 server_base.cc:1034] running on GCE node
W20260619 20:40:07.303928 10578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:07.303913 10577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:07.304311 10572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:07.304572 10572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:07.305763 10572 hybrid_clock.cc:648] HybridClock initialized: now 1781901607305756 us; error 35 us; skew 500 ppm
Jun 19 20:40:07 dist-test-slave-0t40 krb5kdc[10331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901607, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:07.308238 10572 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:40:07.308723 10572 webserver.cc:460] Webserver started at http://127.25.62.130:42703/ using document root <none> and password file <none>
I20260619 20:40:07.308952 10572 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:07.309019 10572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:07.309209 10572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:07.310241 10572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "c1fc159e21d34303881ab58b847b3b1a"
format_stamp: "Formatted at 2026-06-19 20:40:07 on dist-test-slave-0t40"
server_key: "8f79b72597e9c3d3f8c4972104837a13"
server_key_iv: "bd152553186b15ee508f6474791888ac"
server_key_version: "encryptionkey@0"
I20260619 20:40:07.310598 10572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "c1fc159e21d34303881ab58b847b3b1a"
format_stamp: "Formatted at 2026-06-19 20:40:07 on dist-test-slave-0t40"
server_key: "8f79b72597e9c3d3f8c4972104837a13"
server_key_iv: "bd152553186b15ee508f6474791888ac"
server_key_version: "encryptionkey@0"
I20260619 20:40:07.311916 10572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:40:07.312690 10587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:07.312918 10572 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:07.313019 10572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "c1fc159e21d34303881ab58b847b3b1a"
format_stamp: "Formatted at 2026-06-19 20:40:07 on dist-test-slave-0t40"
server_key: "8f79b72597e9c3d3f8c4972104837a13"
server_key_iv: "bd152553186b15ee508f6474791888ac"
server_key_version: "encryptionkey@0"
I20260619 20:40:07.313112 10572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:07.340770 10572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:07.343010 10572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:07.343160 10572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:07.343411 10572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:40:07.343742 10572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:40:07.343801 10572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:07.343842 10572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:40:07.343869 10572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:07.349372 10572 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:44735
I20260619 20:40:07.349421 10700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:44735 every 8 connection(s)
I20260619 20:40:07.349762 10572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:40:07.350569 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 10572
I20260619 20:40:07.350699 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:40:07.350951 25850 external_mini_cluster.cc:1428] Setting key a5539d0fbdc3e9f9d2eebd0b2ea95039
Jun 19 20:40:07 dist-test-slave-0t40 krb5kdc[10331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901607, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.62.131@KRBTEST.COM; defaulting to no policy
I20260619 20:40:07.360057 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.350573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:42425 (local address 127.25.62.190:45871)
0619 20:40:07.350807 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:07.350809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:07.350825 (+ 16us) server_negotiation.cc:407] Connection header received
0619 20:40:07.351464 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:07.351465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:07.351477 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:07.351555 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:07.352535 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.353328 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.354670 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.354876 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.354967 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.357078 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:07.357079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:07.357080 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:07.357088 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:07.358524 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:07.358987 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:07.358988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:07.358989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:07.359047 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:07.359366 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:07.359367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:07.359367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:07.359485 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:07.359561 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:07.359891 (+ 330us) server_negotiation.cc:299] Negotiation successful
0619 20:40:07.359991 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260619 20:40:07.360152 10703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.350525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:45871 (local address 127.25.62.130:42425)
0619 20:40:07.350691 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:07.350734 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:07.351347 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:07.351634 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:07.351638 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:07.352224 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:07.352429 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:07.352434 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.353421 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.353423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:07.354564 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:07.354567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.354990 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.354991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:07.355123 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.355653 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:07.355661 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:07.356969 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:07.358638 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:07.358644 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:07.358646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:07.358901 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:07.359111 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:07.359112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:07.359112 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:07.359282 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:07.359551 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:07.359558 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:07.359700 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 20:40:07.359824 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 20:40:07.359964 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260619 20:40:07.360601 10701 heartbeater.cc:347] Connected to a master server at 127.25.62.190:45871
I20260619 20:40:07.360694 10701 heartbeater.cc:464] Registering TS with master...
I20260619 20:40:07.360863 10701 heartbeater.cc:511] Master 127.25.62.190:45871 requested a full tablet report, sending...
I20260619 20:40:07.361266 10376 ts_manager.cc:194] Registered new tserver with Master: c1fc159e21d34303881ab58b847b3b1a (127.25.62.130:44735)
I20260619 20:40:07.361618 10376 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:42425
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
I20260619 20:40:07.376139 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:45871
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:39609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:07.457681 10706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:07.457899 10706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:07.457926 10706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:07.459173 10706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:07.459223 10706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:07.459251 10706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:40:07.460740 10706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:39609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:45871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:07.461002 10706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:07.461259 10706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:07.466355 10714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:07.466373 10706 server_base.cc:1034] running on GCE node
W20260619 20:40:07.466356 10712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:07.466373 10711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:07.466742 10706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:07.467073 10706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:07.468240 10706 hybrid_clock.cc:648] HybridClock initialized: now 1781901607468218 us; error 34 us; skew 500 ppm
Jun 19 20:40:07 dist-test-slave-0t40 krb5kdc[10331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901607, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:07.470836 10706 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:40:07.471215 10706 webserver.cc:460] Webserver started at http://127.25.62.131:42201/ using document root <none> and password file <none>
I20260619 20:40:07.471473 10706 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:07.471542 10706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:07.471692 10706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:07.472718 10706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "3149ad9c46454d3b8f4cee6d13719b96"
format_stamp: "Formatted at 2026-06-19 20:40:07 on dist-test-slave-0t40"
server_key: "76988c096e5d5a11c17256b3e63ca381"
server_key_iv: "00e9c5f45079ece2f2ad20bf5ad873c8"
server_key_version: "encryptionkey@0"
I20260619 20:40:07.473137 10706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "3149ad9c46454d3b8f4cee6d13719b96"
format_stamp: "Formatted at 2026-06-19 20:40:07 on dist-test-slave-0t40"
server_key: "76988c096e5d5a11c17256b3e63ca381"
server_key_iv: "00e9c5f45079ece2f2ad20bf5ad873c8"
server_key_version: "encryptionkey@0"
I20260619 20:40:07.474468 10706 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:40:07.475174 10721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:07.475358 10706 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:40:07.475452 10706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "3149ad9c46454d3b8f4cee6d13719b96"
format_stamp: "Formatted at 2026-06-19 20:40:07 on dist-test-slave-0t40"
server_key: "76988c096e5d5a11c17256b3e63ca381"
server_key_iv: "00e9c5f45079ece2f2ad20bf5ad873c8"
server_key_version: "encryptionkey@0"
I20260619 20:40:07.475536 10706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:07.501116 10706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:07.503803 10706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:07.503969 10706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:07.504226 10706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:40:07.504570 10706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:40:07.504607 10706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:07.504669 10706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:40:07.504714 10706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:07.511157 10706 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:37327
I20260619 20:40:07.511214 10834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:37327 every 8 connection(s)
I20260619 20:40:07.511539 10706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:40:07 dist-test-slave-0t40 krb5kdc[10331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901607, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:40:07.521279 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 10706
I20260619 20:40:07.521302 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.512259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:43293 (local address 127.25.62.190:45871)
0619 20:40:07.512341 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:07.512345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:07.512532 (+ 187us) server_negotiation.cc:407] Connection header received
0619 20:40:07.513028 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:07.513029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:07.513088 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:07.513157 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:07.514069 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.514598 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.515571 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.515930 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.516002 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.518535 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:07.518536 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:07.518537 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:07.518543 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:07.519785 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:07.520141 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:07.520141 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:07.520142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:07.520194 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:07.520507 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:07.520508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:07.520508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:07.520637 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:07.520814 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:07.521096 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 20:40:07.521190 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 20:40:07.521390 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:40:07.521399 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.512231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:45871 (local address 127.25.62.131:43293)
0619 20:40:07.512410 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:07.512448 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:07.512923 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:07.513214 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:07.513217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:07.513768 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:07.513966 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:07.513971 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.514680 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.514681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:07.515483 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:07.515485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.516057 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.516057 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:07.516170 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.516868 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:07.516878 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:07.518446 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:07.519860 (+ 1414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:07.519865 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:07.519866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:07.520075 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:07.520258 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:07.520258 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:07.520259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:07.520400 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:07.520798 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:07.520803 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:07.520934 (+ 131us) client_negotiation.cc:769] Sending connection context
0619 20:40:07.521020 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 20:40:07.521203 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":103,"threads_started":1}
I20260619 20:40:07.521613 25850 external_mini_cluster.cc:1428] Setting key 5cb2a6234477703beb587c99cc1689ab
I20260619 20:40:07.522192 10835 heartbeater.cc:347] Connected to a master server at 127.25.62.190:45871
I20260619 20:40:07.522310 10835 heartbeater.cc:464] Registering TS with master...
I20260619 20:40:07.522485 10835 heartbeater.cc:511] Master 127.25.62.190:45871 requested a full tablet report, sending...
I20260619 20:40:07.522917 10375 ts_manager.cc:194] Registered new tserver with Master: 3149ad9c46454d3b8f4cee6d13719b96 (127.25.62.131:37327)
I20260619 20:40:07.523334 10375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:43293
I20260619 20:40:07.533499 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:40:07.540800 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.534872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35548 (local address 127.25.62.190:45871)
0619 20:40:07.534927 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:07.534928 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:07.535063 (+ 135us) server_negotiation.cc:407] Connection header received
0619 20:40:07.535111 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:07.535111 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:07.535121 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:07.535175 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:07.535691 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.536161 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.536910 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.537169 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.537223 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.538413 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:07.538414 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:07.538414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:07.538419 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:07.539773 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:07.540132 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:07.540133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:07.540133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:07.540184 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:07.540326 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:07.540326 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:07.540326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:07.540460 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:07.540534 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:07.540676 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 20:40:07.540734 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 20:40:07.542652 10375 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:40:07.544250 10375 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:40:07.550866 10848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.546811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51928 (local address 127.25.62.131:37327)
0619 20:40:07.547208 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:07.547211 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:07.547220 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:40:07.547252 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:07.547255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:07.547271 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:07.547347 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:07.547613 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.548704 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.549958 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.550422 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.550517 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.550576 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:07.550730 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 20:40:07.550776 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":79,"threads_started":1}
I20260619 20:40:07.550851 10847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.546688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:44735 (local address 127.0.0.1:55844)
0619 20:40:07.546856 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:07.546867 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:07.546918 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:07.547361 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:07.547365 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:07.547367 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:07.547429 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:07.547431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.548432 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.548433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:07.549653 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:07.549655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.550552 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.550553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:07.550618 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.550623 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:40:07.550658 (+ 35us) client_negotiation.cc:240] Negotiation successful
0619 20:40:07.550748 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":84,"threads_started":1}
I20260619 20:40:07.550861 10846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.546684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:37327 (local address 127.0.0.1:51928)
0619 20:40:07.546802 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:07.546841 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:07.546905 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:07.547481 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:07.547482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:07.547484 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:07.547530 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:07.547532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.548817 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.548818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:07.549870 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:07.549872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.550521 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.550521 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:07.550601 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.550614 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 20:40:07.550713 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 20:40:07.550778 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":44,"thread_start_us":72,"threads_started":1}
I20260619 20:40:07.551002 10849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.546902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:33247 (local address 127.0.0.1:42842)
0619 20:40:07.547103 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:07.547113 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:07.547157 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:07.547470 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:07.547471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:07.547472 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:07.547517 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:07.547518 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.548864 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.548865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:07.549908 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:07.549911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.550806 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.550807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:07.550864 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.550868 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:40:07.550899 (+ 31us) client_negotiation.cc:240] Negotiation successful
0619 20:40:07.550938 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":1,"thread_start_us":67,"threads_started":1}
I20260619 20:40:07.551293 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.547002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55844 (local address 127.25.62.130:44735)
0619 20:40:07.547158 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:07.547163 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:07.547174 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:40:07.547215 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:07.547218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:07.547234 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:07.547511 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:07.547581 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.548320 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.549744 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.550449 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.550980 (+ 531us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.551136 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:07.551164 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 20:40:07.551212 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260619 20:40:07.551452 10851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.547125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42842 (local address 127.25.62.129:33247)
0619 20:40:07.547296 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:07.547300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:07.547332 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:40:07.547371 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:07.547374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:07.547390 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:07.547462 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:07.547585 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.548711 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.550004 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.550716 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.551241 (+ 525us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.551309 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:07.551335 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 20:40:07.551379 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":109,"threads_started":1}
I20260619 20:40:07.552155 10501 tablet_service.cc:1467] Processing CreateTablet for tablet 6f44335c424747baa0cc21a924fe4c95 (DEFAULT_TABLE table=test-table [id=be8a93c2574746f8bef9029638526c2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:40:07.552174 10635 tablet_service.cc:1467] Processing CreateTablet for tablet 6f44335c424747baa0cc21a924fe4c95 (DEFAULT_TABLE table=test-table [id=be8a93c2574746f8bef9029638526c2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:40:07.552246 10769 tablet_service.cc:1467] Processing CreateTablet for tablet 6f44335c424747baa0cc21a924fe4c95 (DEFAULT_TABLE table=test-table [id=be8a93c2574746f8bef9029638526c2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:40:07.552518 10501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f44335c424747baa0cc21a924fe4c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:07.552521 10769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f44335c424747baa0cc21a924fe4c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:07.552522 10635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f44335c424747baa0cc21a924fe4c95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:07.555424 10852 tablet_bootstrap.cc:492] T 6f44335c424747baa0cc21a924fe4c95 P 8970135a18a6419eb3519b6526172148: Bootstrap starting.
I20260619 20:40:07.555759 10854 tablet_bootstrap.cc:492] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96: Bootstrap starting.
I20260619 20:40:07.556283 10852 tablet_bootstrap.cc:654] T 6f44335c424747baa0cc21a924fe4c95 P 8970135a18a6419eb3519b6526172148: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:07.556423 10854 tablet_bootstrap.cc:654] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:07.556957 10852 log.cc:824] T 6f44335c424747baa0cc21a924fe4c95 P 8970135a18a6419eb3519b6526172148: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:07.557003 10854 log.cc:824] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:07.557252 10853 tablet_bootstrap.cc:492] T 6f44335c424747baa0cc21a924fe4c95 P c1fc159e21d34303881ab58b847b3b1a: Bootstrap starting.
I20260619 20:40:07.557776 10852 tablet_bootstrap.cc:492] T 6f44335c424747baa0cc21a924fe4c95 P 8970135a18a6419eb3519b6526172148: No bootstrap required, opened a new log
I20260619 20:40:07.557853 10852 ts_tablet_manager.cc:1397] T 6f44335c424747baa0cc21a924fe4c95 P 8970135a18a6419eb3519b6526172148: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260619 20:40:07.557938 10853 tablet_bootstrap.cc:654] T 6f44335c424747baa0cc21a924fe4c95 P c1fc159e21d34303881ab58b847b3b1a: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:07.558109 10854 tablet_bootstrap.cc:492] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96: No bootstrap required, opened a new log
I20260619 20:40:07.558185 10854 ts_tablet_manager.cc:1397] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:40:07.558538 10853 log.cc:824] T 6f44335c424747baa0cc21a924fe4c95 P c1fc159e21d34303881ab58b847b3b1a: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:07.559284 10853 tablet_bootstrap.cc:492] T 6f44335c424747baa0cc21a924fe4c95 P c1fc159e21d34303881ab58b847b3b1a: No bootstrap required, opened a new log
I20260619 20:40:07.559360 10853 ts_tablet_manager.cc:1397] T 6f44335c424747baa0cc21a924fe4c95 P c1fc159e21d34303881ab58b847b3b1a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:40:07.560128 10854 raft_consensus.cc:348] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3149ad9c46454d3b8f4cee6d13719b96" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 37327 } } peers { permanent_uuid: "c1fc159e21d34303881ab58b847b3b1a" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44735 } } peers { permanent_uuid: "8970135a18a6419eb3519b6526172148" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33247 } }
I20260619 20:40:07.560133 10852 raft_consensus.cc:348] T 6f44335c424747baa0cc21a924fe4c95 P 8970135a18a6419eb3519b6526172148 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3149ad9c46454d3b8f4cee6d13719b96" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 37327 } } peers { permanent_uuid: "c1fc159e21d34303881ab58b847b3b1a" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44735 } } peers { permanent_uuid: "8970135a18a6419eb3519b6526172148" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33247 } }
I20260619 20:40:07.560315 10852 raft_consensus.cc:374] T 6f44335c424747baa0cc21a924fe4c95 P 8970135a18a6419eb3519b6526172148 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:07.560315 10854 raft_consensus.cc:374] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:07.560352 10852 raft_consensus.cc:729] T 6f44335c424747baa0cc21a924fe4c95 P 8970135a18a6419eb3519b6526172148 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8970135a18a6419eb3519b6526172148, State: Initialized, Role: FOLLOWER
I20260619 20:40:07.560352 10854 raft_consensus.cc:729] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3149ad9c46454d3b8f4cee6d13719b96, State: Initialized, Role: FOLLOWER
I20260619 20:40:07.560482 10854 consensus_queue.cc:260] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3149ad9c46454d3b8f4cee6d13719b96" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 37327 } } peers { permanent_uuid: "c1fc159e21d34303881ab58b847b3b1a" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44735 } } peers { permanent_uuid: "8970135a18a6419eb3519b6526172148" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33247 } }
I20260619 20:40:07.560511 10852 consensus_queue.cc:260] T 6f44335c424747baa0cc21a924fe4c95 P 8970135a18a6419eb3519b6526172148 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3149ad9c46454d3b8f4cee6d13719b96" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 37327 } } peers { permanent_uuid: "c1fc159e21d34303881ab58b847b3b1a" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44735 } } peers { permanent_uuid: "8970135a18a6419eb3519b6526172148" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33247 } }
I20260619 20:40:07.560778 10854 ts_tablet_manager.cc:1428] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:40:07.560778 10852 ts_tablet_manager.cc:1428] T 6f44335c424747baa0cc21a924fe4c95 P 8970135a18a6419eb3519b6526172148: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 20:40:07.560930 10567 heartbeater.cc:503] Master 127.25.62.190:45871 was elected leader, sending a full tablet report...
I20260619 20:40:07.560957 10835 heartbeater.cc:503] Master 127.25.62.190:45871 was elected leader, sending a full tablet report...
I20260619 20:40:07.561107 10853 raft_consensus.cc:348] T 6f44335c424747baa0cc21a924fe4c95 P c1fc159e21d34303881ab58b847b3b1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3149ad9c46454d3b8f4cee6d13719b96" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 37327 } } peers { permanent_uuid: "c1fc159e21d34303881ab58b847b3b1a" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44735 } } peers { permanent_uuid: "8970135a18a6419eb3519b6526172148" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33247 } }
I20260619 20:40:07.561285 10853 raft_consensus.cc:374] T 6f44335c424747baa0cc21a924fe4c95 P c1fc159e21d34303881ab58b847b3b1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:07.561328 10853 raft_consensus.cc:729] T 6f44335c424747baa0cc21a924fe4c95 P c1fc159e21d34303881ab58b847b3b1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1fc159e21d34303881ab58b847b3b1a, State: Initialized, Role: FOLLOWER
I20260619 20:40:07.561437 10853 consensus_queue.cc:260] T 6f44335c424747baa0cc21a924fe4c95 P c1fc159e21d34303881ab58b847b3b1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3149ad9c46454d3b8f4cee6d13719b96" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 37327 } } peers { permanent_uuid: "c1fc159e21d34303881ab58b847b3b1a" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44735 } } peers { permanent_uuid: "8970135a18a6419eb3519b6526172148" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33247 } }
I20260619 20:40:07.561743 10701 heartbeater.cc:503] Master 127.25.62.190:45871 was elected leader, sending a full tablet report...
I20260619 20:40:07.561699 10853 ts_tablet_manager.cc:1428] T 6f44335c424747baa0cc21a924fe4c95 P c1fc159e21d34303881ab58b847b3b1a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260619 20:40:07.600339 10702 tablet.cc:2369] T 6f44335c424747baa0cc21a924fe4c95 P c1fc159e21d34303881ab58b847b3b1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:40:07.610064 10859 raft_consensus.cc:482] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:40:07.610172 10859 raft_consensus.cc:504] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3149ad9c46454d3b8f4cee6d13719b96" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 37327 } } peers { permanent_uuid: "c1fc159e21d34303881ab58b847b3b1a" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44735 } } peers { permanent_uuid: "8970135a18a6419eb3519b6526172148" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33247 } }
I20260619 20:40:07.610675 10859 leader_election.cc:288] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1fc159e21d34303881ab58b847b3b1a (127.25.62.130:44735), 8970135a18a6419eb3519b6526172148 (127.25.62.129:33247)
I20260619 20:40:07.614655 10851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.610931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:47677 (local address 127.25.62.129:33247)
0619 20:40:07.610982 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:07.610984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:07.610997 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:40:07.611027 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:07.611028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:07.611040 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:07.611135 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:07.611323 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.612090 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.613673 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.614279 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.614359 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.614368 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:07.614540 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 20:40:07.614581 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 20:40:07.614658 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.610925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:39913 (local address 127.25.62.130:44735)
0619 20:40:07.610966 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:07.610968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:07.611120 (+ 152us) server_negotiation.cc:407] Connection header received
0619 20:40:07.611174 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:07.611175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:07.611186 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:07.611245 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:07.611405 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.612099 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.613673 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.614271 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.614359 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.614368 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:07.614540 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 20:40:07.614584 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:40:07.614677 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.610844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:33247 (local address 127.25.62.131:47677)
0619 20:40:07.610903 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:07.610914 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:07.610959 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:07.611137 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:07.611138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:07.611139 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:07.611218 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:07.611220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.612182 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.612183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:07.613570 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:07.613573 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.614366 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.614367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:07.614446 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.614448 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:40:07.614525 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 20:40:07.614555 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":114}
I20260619 20:40:07.614688 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.610871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:44735 (local address 127.25.62.131:39913)
0619 20:40:07.611044 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:07.611052 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:07.611120 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:07.611265 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:07.611266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:07.611268 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:07.611338 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:07.611339 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.612182 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.612183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:07.613570 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:07.613573 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.614366 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.614367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:07.614443 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.614447 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:40:07.614514 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 20:40:07.614559 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":65,"thread_start_us":121,"threads_started":1}
I20260619 20:40:07.615131 10521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f44335c424747baa0cc21a924fe4c95" candidate_uuid: "3149ad9c46454d3b8f4cee6d13719b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8970135a18a6419eb3519b6526172148" is_pre_election: true
I20260619 20:40:07.615180 10655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f44335c424747baa0cc21a924fe4c95" candidate_uuid: "3149ad9c46454d3b8f4cee6d13719b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1fc159e21d34303881ab58b847b3b1a" is_pre_election: true
I20260619 20:40:07.615300 10521 raft_consensus.cc:2454] T 6f44335c424747baa0cc21a924fe4c95 P 8970135a18a6419eb3519b6526172148 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3149ad9c46454d3b8f4cee6d13719b96 in term 0.
I20260619 20:40:07.615370 10655 raft_consensus.cc:2454] T 6f44335c424747baa0cc21a924fe4c95 P c1fc159e21d34303881ab58b847b3b1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3149ad9c46454d3b8f4cee6d13719b96 in term 0.
I20260619 20:40:07.615540 10723 leader_election.cc:302] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3149ad9c46454d3b8f4cee6d13719b96, 8970135a18a6419eb3519b6526172148; no voters:
I20260619 20:40:07.615686 10859 raft_consensus.cc:2781] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:40:07.615777 10859 raft_consensus.cc:482] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:40:07.615814 10859 raft_consensus.cc:3037] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:07.616582 10859 raft_consensus.cc:504] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3149ad9c46454d3b8f4cee6d13719b96" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 37327 } } peers { permanent_uuid: "c1fc159e21d34303881ab58b847b3b1a" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44735 } } peers { permanent_uuid: "8970135a18a6419eb3519b6526172148" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33247 } }
I20260619 20:40:07.616747 10859 leader_election.cc:288] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [CANDIDATE]: Term 1 election: Requested vote from peers c1fc159e21d34303881ab58b847b3b1a (127.25.62.130:44735), 8970135a18a6419eb3519b6526172148 (127.25.62.129:33247)
I20260619 20:40:07.616915 10521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f44335c424747baa0cc21a924fe4c95" candidate_uuid: "3149ad9c46454d3b8f4cee6d13719b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8970135a18a6419eb3519b6526172148"
I20260619 20:40:07.616928 10655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f44335c424747baa0cc21a924fe4c95" candidate_uuid: "3149ad9c46454d3b8f4cee6d13719b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1fc159e21d34303881ab58b847b3b1a"
I20260619 20:40:07.616999 10655 raft_consensus.cc:3037] T 6f44335c424747baa0cc21a924fe4c95 P c1fc159e21d34303881ab58b847b3b1a [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:07.617002 10521 raft_consensus.cc:3037] T 6f44335c424747baa0cc21a924fe4c95 P 8970135a18a6419eb3519b6526172148 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:07.617913 10521 raft_consensus.cc:2454] T 6f44335c424747baa0cc21a924fe4c95 P 8970135a18a6419eb3519b6526172148 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3149ad9c46454d3b8f4cee6d13719b96 in term 1.
I20260619 20:40:07.617913 10655 raft_consensus.cc:2454] T 6f44335c424747baa0cc21a924fe4c95 P c1fc159e21d34303881ab58b847b3b1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3149ad9c46454d3b8f4cee6d13719b96 in term 1.
I20260619 20:40:07.618140 10722 leader_election.cc:302] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3149ad9c46454d3b8f4cee6d13719b96, c1fc159e21d34303881ab58b847b3b1a; no voters:
I20260619 20:40:07.618242 10859 raft_consensus.cc:2781] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:07.618414 10859 raft_consensus.cc:686] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [term 1 LEADER]: Becoming Leader. State: Replica: 3149ad9c46454d3b8f4cee6d13719b96, State: Running, Role: LEADER
I20260619 20:40:07.618518 10859 consensus_queue.cc:237] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3149ad9c46454d3b8f4cee6d13719b96" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 37327 } } peers { permanent_uuid: "c1fc159e21d34303881ab58b847b3b1a" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44735 } } peers { permanent_uuid: "8970135a18a6419eb3519b6526172148" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33247 } }
I20260619 20:40:07.619423 10376 catalog_manager.cc:5495] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3149ad9c46454d3b8f4cee6d13719b96 (127.25.62.131). New cstate: current_term: 1 leader_uuid: "3149ad9c46454d3b8f4cee6d13719b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3149ad9c46454d3b8f4cee6d13719b96" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 37327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1fc159e21d34303881ab58b847b3b1a" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8970135a18a6419eb3519b6526172148" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33247 } health_report { overall_health: UNKNOWN } } }
I20260619 20:40:07.642249 10848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.639852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51930 (local address 127.25.62.131:37327)
0619 20:40:07.639910 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:07.639912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:07.639921 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:40:07.639962 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:07.639963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:07.639978 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:07.640038 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:40:07.640220 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.640673 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.641515 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.641703 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.641764 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.641936 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:40:07.642044 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:40:07.642124 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:07.642162 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 20:40:07.642184 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 20:40:07.644126 10521 raft_consensus.cc:1261] T 6f44335c424747baa0cc21a924fe4c95 P 8970135a18a6419eb3519b6526172148 [term 1 FOLLOWER]: Refusing update from remote peer 3149ad9c46454d3b8f4cee6d13719b96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:40:07.644129 10655 raft_consensus.cc:1261] T 6f44335c424747baa0cc21a924fe4c95 P c1fc159e21d34303881ab58b847b3b1a [term 1 FOLLOWER]: Refusing update from remote peer 3149ad9c46454d3b8f4cee6d13719b96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:40:07.644531 10862 consensus_queue.cc:1035] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8970135a18a6419eb3519b6526172148" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:40:07.644677 10859 consensus_queue.cc:1035] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1fc159e21d34303881ab58b847b3b1a" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 44735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:40:07.647136 10865 mvcc.cc:205] Tried to move back new op lower bound from 7298668984907464704 to 7298668984806199296. Current Snapshot: MvccSnapshot[applied={T|T < 7298668984907464704}]
I20260619 20:40:07.647425 10866 mvcc.cc:205] Tried to move back new op lower bound from 7298668984907464704 to 7298668984806199296. Current Snapshot: MvccSnapshot[applied={T|T < 7298668984907464704}]
I20260619 20:40:07.649282 10376 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:40:07.649384 10376 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:40:07.650167 10376 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff2fa5271dec4c9e8ee52f657172324f: Sending DeleteTablet for 3 replicas of tablet 6f44335c424747baa0cc21a924fe4c95
I20260619 20:40:07.650501 10635 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f44335c424747baa0cc21a924fe4c95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:40:07 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:55844
I20260619 20:40:07.650594 10769 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f44335c424747baa0cc21a924fe4c95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:40:07 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:51928
I20260619 20:40:07.650804 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 10438
I20260619 20:40:07.650851 10501 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f44335c424747baa0cc21a924fe4c95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:40:07 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:42842
I20260619 20:40:07.651364 10876 tablet_replica.cc:330] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96: stopping tablet replica
I20260619 20:40:07.651469 10876 raft_consensus.cc:2229] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:40:07.651573 10876 raft_consensus.cc:2258] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:40:07.652010 10876 ts_tablet_manager.cc:1905] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:40:07.654270 10876 ts_tablet_manager.cc:1918] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:40:07.654361 10876 log.cc:1196] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/6f44335c424747baa0cc21a924fe4c95
I20260619 20:40:07.654685 10876 ts_tablet_manager.cc:1939] T 6f44335c424747baa0cc21a924fe4c95 P 3149ad9c46454d3b8f4cee6d13719b96: Deleting consensus metadata
I20260619 20:40:07.655148 10361 catalog_manager.cc:4841] TS 3149ad9c46454d3b8f4cee6d13719b96 (127.25.62.131:37327): tablet 6f44335c424747baa0cc21a924fe4c95 (table test-table [id=be8a93c2574746f8bef9029638526c2c]) successfully deleted
I20260619 20:40:07.657196 10875 tablet_replica.cc:330] T 6f44335c424747baa0cc21a924fe4c95 P c1fc159e21d34303881ab58b847b3b1a: stopping tablet replica
I20260619 20:40:07.657325 10875 raft_consensus.cc:2229] T 6f44335c424747baa0cc21a924fe4c95 P c1fc159e21d34303881ab58b847b3b1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:40:07.657444 10875 raft_consensus.cc:2258] T 6f44335c424747baa0cc21a924fe4c95 P c1fc159e21d34303881ab58b847b3b1a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 20:40:07.658012 10361 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.62.129:33247 (error 108)
I20260619 20:40:07.658227 10875 ts_tablet_manager.cc:1905] T 6f44335c424747baa0cc21a924fe4c95 P c1fc159e21d34303881ab58b847b3b1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:40:07.658882 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 10572
I20260619 20:40:07.658911 10846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.658762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:33247 (local address 127.0.0.1:42846)
0619 20:40:07.658818 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:07.658864 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.129:33247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 20:40:07.661373 10361 catalog_manager.cc:4584] TS 8970135a18a6419eb3519b6526172148 (127.25.62.129:33247): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f44335c424747baa0cc21a924fe4c95: Network error: Client connection negotiation failed: client connection to 127.25.62.129:33247: connect: Connection refused (error 111)
I20260619 20:40:07.664783 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 10706
I20260619 20:40:07.664923 10846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.664789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:44735 (local address 127.0.0.1:55846)
0619 20:40:07.664849 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:07.664885 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.130:44735: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 20:40:07.669731 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 10344
2026-06-19T20:40:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1991 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T20:40:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:07Z Disabled control of system clock
I20260619 20:40:07.696527 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:46347
--webserver_interface=127.25.62.190
--webserver_port=0
--builtin_ntp_servers=127.25.62.148:42147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:46347
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:40:07.775188 10881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:07.775391 10881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:07.775411 10881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:07.776453 10881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:07.776484 10881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:07.776497 10881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:07.776510 10881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:07.776521 10881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:07.778039 10881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:42147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:46347
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:46347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:07.778240 10881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:07.778467 10881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:40:07.783211 10881 server_base.cc:1034] running on GCE node
W20260619 20:40:07.783385 10887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:07.783437 10886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:07.783510 10889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:07.783762 10881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:07.784060 10881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:07.785220 10881 hybrid_clock.cc:648] HybridClock initialized: now 1781901607785202 us; error 31 us; skew 500 ppm
I20260619 20:40:07.786419 10881 webserver.cc:460] Webserver started at http://127.25.62.190:43783/ using document root <none> and password file <none>
I20260619 20:40:07.786688 10881 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:07.786760 10881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:07.786916 10881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:07.787958 10881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "413cdf5e690648a785a6b6aef677ed8e"
format_stamp: "Formatted at 2026-06-19 20:40:07 on dist-test-slave-0t40"
server_key: "19a4f8585ac2fa772d502d5f4889594a"
server_key_iv: "78a0e33af31e3b0a7e1173326d10beee"
server_key_version: "encryptionkey@0"
I20260619 20:40:07.788321 10881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "413cdf5e690648a785a6b6aef677ed8e"
format_stamp: "Formatted at 2026-06-19 20:40:07 on dist-test-slave-0t40"
server_key: "19a4f8585ac2fa772d502d5f4889594a"
server_key_iv: "78a0e33af31e3b0a7e1173326d10beee"
server_key_version: "encryptionkey@0"
I20260619 20:40:07.789680 10881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:40:07.790393 10895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:07.790558 10881 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:40:07.790660 10881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "413cdf5e690648a785a6b6aef677ed8e"
format_stamp: "Formatted at 2026-06-19 20:40:07 on dist-test-slave-0t40"
server_key: "19a4f8585ac2fa772d502d5f4889594a"
server_key_iv: "78a0e33af31e3b0a7e1173326d10beee"
server_key_version: "encryptionkey@0"
I20260619 20:40:07.790732 10881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:07.803938 10881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:07.804299 10881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:07.804464 10881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:07.809353 10881 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:46347
I20260619 20:40:07.809785 10881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:07.810485 10947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:46347 every 8 connection(s)
I20260619 20:40:07.811483 10948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:07.815099 10948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e: Bootstrap starting.
I20260619 20:40:07.815822 10948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:07.816151 10948 log.cc:824] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:07.816632 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 10881
I20260619 20:40:07.816725 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:07.816923 25850 external_mini_cluster.cc:1428] Setting key 338ed27270e8d05d077a077562a37360
I20260619 20:40:07.816980 10948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e: No bootstrap required, opened a new log
I20260619 20:40:07.819774 10948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413cdf5e690648a785a6b6aef677ed8e" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46347 } }
I20260619 20:40:07.819969 10948 raft_consensus.cc:374] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:07.820011 10948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413cdf5e690648a785a6b6aef677ed8e, State: Initialized, Role: FOLLOWER
I20260619 20:40:07.820161 10948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413cdf5e690648a785a6b6aef677ed8e" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46347 } }
I20260619 20:40:07.820266 10948 raft_consensus.cc:388] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:07.820298 10948 raft_consensus.cc:482] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:07.820356 10948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:07.821015 10948 raft_consensus.cc:504] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413cdf5e690648a785a6b6aef677ed8e" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46347 } }
I20260619 20:40:07.821163 10948 leader_election.cc:302] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 413cdf5e690648a785a6b6aef677ed8e; no voters:
I20260619 20:40:07.821326 10948 leader_election.cc:288] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:07.821427 10953 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:07.821599 10953 raft_consensus.cc:686] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e [term 1 LEADER]: Becoming Leader. State: Replica: 413cdf5e690648a785a6b6aef677ed8e, State: Running, Role: LEADER
I20260619 20:40:07.821607 10948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:07.821780 10953 consensus_queue.cc:237] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413cdf5e690648a785a6b6aef677ed8e" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46347 } }
I20260619 20:40:07.822155 10953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 413cdf5e690648a785a6b6aef677ed8e. Latest consensus state: current_term: 1 leader_uuid: "413cdf5e690648a785a6b6aef677ed8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413cdf5e690648a785a6b6aef677ed8e" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46347 } } }
I20260619 20:40:07.822404 10953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:07.822333 10954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "413cdf5e690648a785a6b6aef677ed8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413cdf5e690648a785a6b6aef677ed8e" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 46347 } } }
I20260619 20:40:07.822659 10954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:07.822659 10962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:07.823210 10962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:07.824891 10952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:07.817495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42548 (local address 127.25.62.190:46347)
0619 20:40:07.818122 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:07.818133 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:07.818174 (+ 41us) server_negotiation.cc:407] Connection header received
0619 20:40:07.818821 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:07.818838 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:07.818918 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:07.819069 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:07.819357 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.821258 (+ 1901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.822476 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:07.823135 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:07.823546 (+ 411us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:07.824219 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:07.824223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:07.824223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:07.824433 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:07.824479 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:07.824495 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:07.824560 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:07.824636 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 20:40:07.824712 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":87,"threads_started":1}
I20260619 20:40:07.825502 10962 catalog_manager.cc:1347] Generated new cluster ID: 0da7b841c01743fbab6a104fa6863a1d
I20260619 20:40:07.825548 10962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:07.832860 10962 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:07.833420 10962 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:07.835796 10962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e: Generated new TSK 0
I20260619 20:40:07.835960 10962 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:07.839730 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:46347
--builtin_ntp_servers=127.25.62.148:42147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:40:07.948870 10972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:07.949131 10972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:07.949157 10972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:07.950372 10972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:07.950414 10972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:07.950440 10972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:40:07.951851 10972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:42147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:46347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:07.952117 10972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:07.952466 10972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:40:07.957581 10972 server_base.cc:1034] running on GCE node
W20260619 20:40:07.957557 10977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:07.957648 10978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:07.957832 10980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:07.958168 10972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:07.958454 10972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:07.959609 10972 hybrid_clock.cc:648] HybridClock initialized: now 1781901607959584 us; error 43 us; skew 500 ppm
I20260619 20:40:07.960878 10972 webserver.cc:460] Webserver started at http://127.25.62.129:34431/ using document root <none> and password file <none>
I20260619 20:40:07.961151 10972 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:07.961199 10972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:07.961302 10972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:07.962359 10972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "6c5df6fdf5004c6fad622537207f4344"
format_stamp: "Formatted at 2026-06-19 20:40:07 on dist-test-slave-0t40"
server_key: "bc42b82ae671ee8d767f1f3d2db03a91"
server_key_iv: "d67b62df53370964cacb1262da38c598"
server_key_version: "encryptionkey@0"
I20260619 20:40:07.962668 10972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "6c5df6fdf5004c6fad622537207f4344"
format_stamp: "Formatted at 2026-06-19 20:40:07 on dist-test-slave-0t40"
server_key: "bc42b82ae671ee8d767f1f3d2db03a91"
server_key_iv: "d67b62df53370964cacb1262da38c598"
server_key_version: "encryptionkey@0"
I20260619 20:40:07.963869 10972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:40:07.964474 10986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:07.964627 10972 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:40:07.964710 10972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "6c5df6fdf5004c6fad622537207f4344"
format_stamp: "Formatted at 2026-06-19 20:40:07 on dist-test-slave-0t40"
server_key: "bc42b82ae671ee8d767f1f3d2db03a91"
server_key_iv: "d67b62df53370964cacb1262da38c598"
server_key_version: "encryptionkey@0"
I20260619 20:40:07.964766 10972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:07.993103 10972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:07.993408 10972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:07.993526 10972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:07.993705 10972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:40:07.994041 10972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:40:07.994076 10972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:07.994098 10972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:40:07.994112 10972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:08.000066 10972 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:35517
I20260619 20:40:08.000219 11099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:35517 every 8 connection(s)
I20260619 20:40:08.000526 10972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:40:08.005713 10952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:08.001371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:33347 (local address 127.25.62.190:46347)
0619 20:40:08.001454 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:08.001456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:08.001664 (+ 208us) server_negotiation.cc:407] Connection header received
0619 20:40:08.002165 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:08.002166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:08.002181 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:08.002235 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:08.002893 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.003401 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.004368 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.004555 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.004609 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:08.005481 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:08.005482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:08.005482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:08.005486 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:08.005499 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:08.005502 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:08.005546 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:08.005632 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 20:40:08.005653 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 20:40:08.005957 11102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:08.001336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46347 (local address 127.25.62.129:33347)
0619 20:40:08.001542 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:08.001577 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:08.002082 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:08.002291 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:08.002295 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:08.002303 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:08.002812 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:08.002816 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.003484 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.003485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:08.004281 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:08.004283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.004636 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.004637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:08.004763 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:08.005327 (+ 564us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:40:08.005328 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:08.005356 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:40:08.005386 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:08.005560 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:08.005565 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:08.005567 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:40:08.005619 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 20:40:08.005699 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":102,"threads_started":1}
I20260619 20:40:08.006518 11100 heartbeater.cc:347] Connected to a master server at 127.25.62.190:46347
I20260619 20:40:08.006624 11100 heartbeater.cc:464] Registering TS with master...
I20260619 20:40:08.006805 11100 heartbeater.cc:511] Master 127.25.62.190:46347 requested a full tablet report, sending...
I20260619 20:40:08.007323 10904 ts_manager.cc:194] Registered new tserver with Master: 6c5df6fdf5004c6fad622537207f4344 (127.25.62.129:35517)
I20260619 20:40:08.008028 10904 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.62.129:33347
I20260619 20:40:08.008935 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 10972
I20260619 20:40:08.009078 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:40:08.009282 25850 external_mini_cluster.cc:1428] Setting key 96689200cc5bc4a75c553517079a10bb
I20260619 20:40:08.010310 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:46347
--builtin_ntp_servers=127.25.62.148:42147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:40:08.095005 11103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:08.095240 11103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:08.095279 11103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:08.096406 11103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:08.096474 11103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:08.096518 11103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:40:08.097886 11103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:42147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:46347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:08.098161 11103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:08.098469 11103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:08.103453 11109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:08.103489 11103 server_base.cc:1034] running on GCE node
W20260619 20:40:08.103524 11111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:08.103717 11108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:08.103976 11103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:08.104236 11103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:08.105398 11103 hybrid_clock.cc:648] HybridClock initialized: now 1781901608105384 us; error 45 us; skew 500 ppm
I20260619 20:40:08.106652 11103 webserver.cc:460] Webserver started at http://127.25.62.130:38199/ using document root <none> and password file <none>
I20260619 20:40:08.106921 11103 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:08.106997 11103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:08.107148 11103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:08.108176 11103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "c2d8741a94af410da9c8c37595b0617b"
format_stamp: "Formatted at 2026-06-19 20:40:08 on dist-test-slave-0t40"
server_key: "dfc8655041f944bf33c598ce4db1db70"
server_key_iv: "4fce60dc1df01542c8f0221b0725fb93"
server_key_version: "encryptionkey@0"
I20260619 20:40:08.108546 11103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "c2d8741a94af410da9c8c37595b0617b"
format_stamp: "Formatted at 2026-06-19 20:40:08 on dist-test-slave-0t40"
server_key: "dfc8655041f944bf33c598ce4db1db70"
server_key_iv: "4fce60dc1df01542c8f0221b0725fb93"
server_key_version: "encryptionkey@0"
I20260619 20:40:08.109934 11103 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:40:08.110662 11117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:08.110819 11103 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:40:08.110920 11103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "c2d8741a94af410da9c8c37595b0617b"
format_stamp: "Formatted at 2026-06-19 20:40:08 on dist-test-slave-0t40"
server_key: "dfc8655041f944bf33c598ce4db1db70"
server_key_iv: "4fce60dc1df01542c8f0221b0725fb93"
server_key_version: "encryptionkey@0"
I20260619 20:40:08.111002 11103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:08.125643 11103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:08.125937 11103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:08.126093 11103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:08.126318 11103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:40:08.126641 11103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:40:08.126698 11103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:08.126747 11103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:40:08.126777 11103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:08.133428 11103 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:42557
I20260619 20:40:08.133487 11230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:42557 every 8 connection(s)
I20260619 20:40:08.133816 11103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:40:08.138955 10952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:08.134485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:35635 (local address 127.25.62.190:46347)
0619 20:40:08.134546 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:08.134549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:08.134784 (+ 235us) server_negotiation.cc:407] Connection header received
0619 20:40:08.135303 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:08.135305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:08.135317 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:08.135380 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:08.136028 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.136531 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.137568 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.137766 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.137883 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:08.138654 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:08.138656 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:08.138656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:08.138661 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:08.138676 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:08.138680 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:08.138747 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:08.138849 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 20:40:08.138877 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 20:40:08.139159 11233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:08.134476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46347 (local address 127.25.62.130:35635)
0619 20:40:08.134656 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:08.134687 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:08.135183 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:08.135449 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:08.135452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:08.135459 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:08.135935 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:08.135939 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.136622 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.136624 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:08.137469 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:08.137471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.137892 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.137893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:08.138022 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:08.138511 (+ 489us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:40:08.138511 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:08.138538 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:40:08.138558 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:08.138760 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:08.138765 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:08.138767 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:40:08.138853 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 20:40:08.138946 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260619 20:40:08.139652 11231 heartbeater.cc:347] Connected to a master server at 127.25.62.190:46347
I20260619 20:40:08.139750 11231 heartbeater.cc:464] Registering TS with master...
I20260619 20:40:08.139912 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 11103
I20260619 20:40:08.139937 11231 heartbeater.cc:511] Master 127.25.62.190:46347 requested a full tablet report, sending...
I20260619 20:40:08.140012 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:40:08.140251 25850 external_mini_cluster.cc:1428] Setting key f5e24f7a6bd36e9519efb2e4679bf15a
I20260619 20:40:08.140347 10904 ts_manager.cc:194] Registered new tserver with Master: c2d8741a94af410da9c8c37595b0617b (127.25.62.130:42557)
I20260619 20:40:08.140868 10904 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.62.130:35635
I20260619 20:40:08.141448 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:46347
--builtin_ntp_servers=127.25.62.148:42147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:40:08.221433 11234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:08.221714 11234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:08.221766 11234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:08.223060 11234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:08.223116 11234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:08.223156 11234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:40:08.224484 11234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:42147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:46347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:08.224751 11234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:08.225059 11234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:08.229828 11242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:08.229866 11240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:08.229830 11239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:08.230157 11234 server_base.cc:1034] running on GCE node
I20260619 20:40:08.230350 11234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:08.230530 11234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:08.231685 11234 hybrid_clock.cc:648] HybridClock initialized: now 1781901608231661 us; error 37 us; skew 500 ppm
I20260619 20:40:08.232950 11234 webserver.cc:460] Webserver started at http://127.25.62.131:38755/ using document root <none> and password file <none>
I20260619 20:40:08.233218 11234 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:08.233279 11234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:08.233436 11234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:08.234494 11234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "dd92a7ef70654e889444cf1267b8c565"
format_stamp: "Formatted at 2026-06-19 20:40:08 on dist-test-slave-0t40"
server_key: "c788d6cc764754cc7226d33c1df59719"
server_key_iv: "ff328489ba6afe8cbe2e504968c56364"
server_key_version: "encryptionkey@0"
I20260619 20:40:08.234859 11234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "dd92a7ef70654e889444cf1267b8c565"
format_stamp: "Formatted at 2026-06-19 20:40:08 on dist-test-slave-0t40"
server_key: "c788d6cc764754cc7226d33c1df59719"
server_key_iv: "ff328489ba6afe8cbe2e504968c56364"
server_key_version: "encryptionkey@0"
I20260619 20:40:08.236209 11234 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:40:08.236976 11248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:08.237155 11234 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:40:08.237267 11234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "dd92a7ef70654e889444cf1267b8c565"
format_stamp: "Formatted at 2026-06-19 20:40:08 on dist-test-slave-0t40"
server_key: "c788d6cc764754cc7226d33c1df59719"
server_key_iv: "ff328489ba6afe8cbe2e504968c56364"
server_key_version: "encryptionkey@0"
I20260619 20:40:08.237342 11234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:08.252646 11234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:08.252914 11234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:08.253103 11234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:08.253326 11234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:40:08.253636 11234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:40:08.253690 11234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:08.253729 11234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:40:08.253756 11234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:08.259246 11234 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:35761
I20260619 20:40:08.259286 11361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:35761 every 8 connection(s)
I20260619 20:40:08.259588 11234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:40:08.264503 10952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:08.260264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:43219 (local address 127.25.62.190:46347)
0619 20:40:08.260329 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:08.260331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:08.260568 (+ 237us) server_negotiation.cc:407] Connection header received
0619 20:40:08.261075 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:08.261077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:08.261087 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:08.261149 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:08.261786 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.262268 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.263239 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.263436 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.263493 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:08.264291 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:08.264292 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:08.264293 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:08.264296 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:08.264307 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:08.264310 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:08.264350 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:08.264429 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 20:40:08.264446 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 20:40:08.264739 11364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:08.260249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:46347 (local address 127.25.62.131:43219)
0619 20:40:08.260438 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:08.260479 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:08.260947 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:08.261215 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:08.261219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:08.261227 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:08.261698 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:08.261702 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.262371 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.262372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:08.263157 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:08.263160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.263516 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.263517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:08.263618 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:08.264172 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:40:08.264173 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:08.264200 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:40:08.264222 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:08.264364 (+ 142us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:08.264367 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:08.264370 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:40:08.264420 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 20:40:08.264493 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":100,"threads_started":1}
I20260619 20:40:08.265262 11362 heartbeater.cc:347] Connected to a master server at 127.25.62.190:46347
I20260619 20:40:08.265362 11362 heartbeater.cc:464] Registering TS with master...
I20260619 20:40:08.265528 11362 heartbeater.cc:511] Master 127.25.62.190:46347 requested a full tablet report, sending...
I20260619 20:40:08.265915 10904 ts_manager.cc:194] Registered new tserver with Master: dd92a7ef70654e889444cf1267b8c565 (127.25.62.131:35761)
I20260619 20:40:08.266076 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 11234
I20260619 20:40:08.266171 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:40:08.266358 10904 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.62.131:43219
I20260619 20:40:08.266382 25850 external_mini_cluster.cc:1428] Setting key eda2fce65c6d7ee6580cf91637dfbd33
I20260619 20:40:08.267251 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:40:08.271399 10952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:08.268602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42550 (local address 127.25.62.190:46347)
0619 20:40:08.268682 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:08.268683 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:08.268792 (+ 109us) server_negotiation.cc:407] Connection header received
0619 20:40:08.268866 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:08.268867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:08.268876 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:08.268920 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:08.269110 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.269566 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.270317 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.270516 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.270596 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:08.271172 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:08.271173 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:08.271173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:08.271177 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:08.271188 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:08.271192 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:08.271252 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:08.271303 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 20:40:08.271322 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 20:40:08.272774 10904 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:40:08.274331 10904 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:40:08.280305 11373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:08.276472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59778 (local address 127.25.62.130:42557)
0619 20:40:08.276611 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:08.276616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:08.276687 (+ 71us) server_negotiation.cc:407] Connection header received
0619 20:40:08.276765 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:08.276767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:08.276782 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:08.277028 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:08.277111 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.277904 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.279192 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.279924 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.280002 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:08.280089 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:08.280188 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 20:40:08.280224 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
I20260619 20:40:08.280316 11371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:08.276296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:35761 (local address 127.0.0.1:51684)
0619 20:40:08.276455 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:08.276492 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:08.276554 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:08.276866 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:08.276868 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:08.276870 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:08.276938 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:08.276940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.278280 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.278281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:08.279162 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:08.279164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.280025 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.280026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:08.280103 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:08.280107 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:40:08.280143 (+ 36us) client_negotiation.cc:240] Negotiation successful
0619 20:40:08.280196 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":6,"thread_start_us":58,"threads_started":1}
I20260619 20:40:08.280442 11374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:08.276303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:42557 (local address 127.0.0.1:59778)
0619 20:40:08.276633 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:08.276643 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:08.276700 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:08.276883 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:08.276883 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:08.276885 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:08.276949 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:08.276951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.278000 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.278001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:08.279120 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:08.279122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.280011 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.280012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:08.280084 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:08.280096 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 20:40:08.280166 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 20:40:08.280206 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":1,"thread_start_us":74,"threads_started":1}
I20260619 20:40:08.280476 11375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:08.276688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40858 (local address 127.25.62.129:35517)
0619 20:40:08.276853 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:08.276858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:08.276934 (+ 76us) server_negotiation.cc:407] Connection header received
0619 20:40:08.277077 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:08.277079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:08.277095 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:08.277168 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:08.277332 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.278383 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.279484 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.279983 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.280051 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:08.280139 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:08.280356 (+ 217us) server_negotiation.cc:299] Negotiation successful
0619 20:40:08.280399 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260619 20:40:08.280516 11372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:08.276379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51684 (local address 127.25.62.131:35761)
0619 20:40:08.276690 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:08.276694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:08.276704 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:40:08.276740 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:08.276743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:08.276758 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:08.277029 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:08.277110 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.278186 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.279250 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.279923 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.280254 (+ 331us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:08.280348 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:08.280373 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 20:40:08.280416 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":102,"threads_started":1}
I20260619 20:40:08.280594 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:08.276483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:35517 (local address 127.0.0.1:40858)
0619 20:40:08.276879 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:08.276889 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:08.276940 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:08.277230 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:08.277231 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:08.277233 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:08.277266 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:08.277267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.278464 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.278465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:08.279400 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:08.279401 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.280173 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.280174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:08.280224 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:08.280228 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:40:08.280273 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 20:40:08.280306 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":78,"threads_started":1}
I20260619 20:40:08.281549 11034 tablet_service.cc:1467] Processing CreateTablet for tablet 1b5764549a0e46fda975e19c1ba9c455 (DEFAULT_TABLE table=test-table [id=2924fb7a34284a2b893d4370d482d7ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:40:08.281549 11296 tablet_service.cc:1467] Processing CreateTablet for tablet 1b5764549a0e46fda975e19c1ba9c455 (DEFAULT_TABLE table=test-table [id=2924fb7a34284a2b893d4370d482d7ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:40:08.281895 11296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b5764549a0e46fda975e19c1ba9c455. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:08.281895 11034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b5764549a0e46fda975e19c1ba9c455. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:08.284572 11165 tablet_service.cc:1467] Processing CreateTablet for tablet 1b5764549a0e46fda975e19c1ba9c455 (DEFAULT_TABLE table=test-table [id=2924fb7a34284a2b893d4370d482d7ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:40:08.284914 11165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b5764549a0e46fda975e19c1ba9c455. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:08.285161 11377 tablet_bootstrap.cc:492] T 1b5764549a0e46fda975e19c1ba9c455 P 6c5df6fdf5004c6fad622537207f4344: Bootstrap starting.
I20260619 20:40:08.285326 11376 tablet_bootstrap.cc:492] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565: Bootstrap starting.
I20260619 20:40:08.285854 11376 tablet_bootstrap.cc:654] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:08.285869 11377 tablet_bootstrap.cc:654] T 1b5764549a0e46fda975e19c1ba9c455 P 6c5df6fdf5004c6fad622537207f4344: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:08.286149 11376 log.cc:824] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:08.286180 11377 log.cc:824] T 1b5764549a0e46fda975e19c1ba9c455 P 6c5df6fdf5004c6fad622537207f4344: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:08.286808 11376 tablet_bootstrap.cc:492] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565: No bootstrap required, opened a new log
I20260619 20:40:08.286886 11376 ts_tablet_manager.cc:1397] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:40:08.287966 11380 tablet_bootstrap.cc:492] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b: Bootstrap starting.
I20260619 20:40:08.288475 11377 tablet_bootstrap.cc:492] T 1b5764549a0e46fda975e19c1ba9c455 P 6c5df6fdf5004c6fad622537207f4344: No bootstrap required, opened a new log
I20260619 20:40:08.288546 11377 ts_tablet_manager.cc:1397] T 1b5764549a0e46fda975e19c1ba9c455 P 6c5df6fdf5004c6fad622537207f4344: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260619 20:40:08.288574 11380 tablet_bootstrap.cc:654] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:08.288823 11380 log.cc:824] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:08.289489 11380 tablet_bootstrap.cc:492] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b: No bootstrap required, opened a new log
I20260619 20:40:08.289566 11380 ts_tablet_manager.cc:1397] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:40:08.289467 11376 raft_consensus.cc:348] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92a7ef70654e889444cf1267b8c565" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 35761 } } peers { permanent_uuid: "c2d8741a94af410da9c8c37595b0617b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 42557 } } peers { permanent_uuid: "6c5df6fdf5004c6fad622537207f4344" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 35517 } }
I20260619 20:40:08.289618 11376 raft_consensus.cc:374] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:08.289657 11376 raft_consensus.cc:729] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd92a7ef70654e889444cf1267b8c565, State: Initialized, Role: FOLLOWER
I20260619 20:40:08.289810 11376 consensus_queue.cc:260] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92a7ef70654e889444cf1267b8c565" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 35761 } } peers { permanent_uuid: "c2d8741a94af410da9c8c37595b0617b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 42557 } } peers { permanent_uuid: "6c5df6fdf5004c6fad622537207f4344" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 35517 } }
I20260619 20:40:08.290068 11376 ts_tablet_manager.cc:1428] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:40:08.290206 11362 heartbeater.cc:503] Master 127.25.62.190:46347 was elected leader, sending a full tablet report...
I20260619 20:40:08.291107 11377 raft_consensus.cc:348] T 1b5764549a0e46fda975e19c1ba9c455 P 6c5df6fdf5004c6fad622537207f4344 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92a7ef70654e889444cf1267b8c565" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 35761 } } peers { permanent_uuid: "c2d8741a94af410da9c8c37595b0617b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 42557 } } peers { permanent_uuid: "6c5df6fdf5004c6fad622537207f4344" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 35517 } }
I20260619 20:40:08.291263 11377 raft_consensus.cc:374] T 1b5764549a0e46fda975e19c1ba9c455 P 6c5df6fdf5004c6fad622537207f4344 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:08.291321 11377 raft_consensus.cc:729] T 1b5764549a0e46fda975e19c1ba9c455 P 6c5df6fdf5004c6fad622537207f4344 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c5df6fdf5004c6fad622537207f4344, State: Initialized, Role: FOLLOWER
I20260619 20:40:08.291438 11377 consensus_queue.cc:260] T 1b5764549a0e46fda975e19c1ba9c455 P 6c5df6fdf5004c6fad622537207f4344 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92a7ef70654e889444cf1267b8c565" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 35761 } } peers { permanent_uuid: "c2d8741a94af410da9c8c37595b0617b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 42557 } } peers { permanent_uuid: "6c5df6fdf5004c6fad622537207f4344" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 35517 } }
I20260619 20:40:08.291672 11377 ts_tablet_manager.cc:1428] T 1b5764549a0e46fda975e19c1ba9c455 P 6c5df6fdf5004c6fad622537207f4344: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 20:40:08.291749 11100 heartbeater.cc:503] Master 127.25.62.190:46347 was elected leader, sending a full tablet report...
I20260619 20:40:08.291882 11380 raft_consensus.cc:348] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92a7ef70654e889444cf1267b8c565" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 35761 } } peers { permanent_uuid: "c2d8741a94af410da9c8c37595b0617b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 42557 } } peers { permanent_uuid: "6c5df6fdf5004c6fad622537207f4344" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 35517 } }
I20260619 20:40:08.292017 11380 raft_consensus.cc:374] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:08.292059 11380 raft_consensus.cc:729] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2d8741a94af410da9c8c37595b0617b, State: Initialized, Role: FOLLOWER
I20260619 20:40:08.292161 11380 consensus_queue.cc:260] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92a7ef70654e889444cf1267b8c565" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 35761 } } peers { permanent_uuid: "c2d8741a94af410da9c8c37595b0617b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 42557 } } peers { permanent_uuid: "6c5df6fdf5004c6fad622537207f4344" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 35517 } }
I20260619 20:40:08.292399 11380 ts_tablet_manager.cc:1428] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:40:08.292477 11231 heartbeater.cc:503] Master 127.25.62.190:46347 was elected leader, sending a full tablet report...
I20260619 20:40:08.328162 11382 raft_consensus.cc:482] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:40:08.328356 11382 raft_consensus.cc:504] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92a7ef70654e889444cf1267b8c565" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 35761 } } peers { permanent_uuid: "c2d8741a94af410da9c8c37595b0617b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 42557 } } peers { permanent_uuid: "6c5df6fdf5004c6fad622537207f4344" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 35517 } }
I20260619 20:40:08.328756 11382 leader_election.cc:288] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2d8741a94af410da9c8c37595b0617b (127.25.62.130:42557), 6c5df6fdf5004c6fad622537207f4344 (127.25.62.129:35517)
I20260619 20:40:08.331650 11373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:08.329046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:34005 (local address 127.25.62.130:42557)
0619 20:40:08.329108 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:08.329109 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:08.329277 (+ 168us) server_negotiation.cc:407] Connection header received
0619 20:40:08.329313 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:08.329314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:08.329323 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:08.329385 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:08.329554 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.330027 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.330962 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.331307 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.331398 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:08.331406 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:08.331539 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 20:40:08.331576 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:40:08.331650 11375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:08.329067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:43623 (local address 127.25.62.129:35517)
0619 20:40:08.329119 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:08.329121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:08.329133 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:40:08.329168 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:08.329169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:08.329179 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:08.329249 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:08.329438 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.329958 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.330912 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.331280 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.331341 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:08.331348 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:08.331536 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 20:40:08.331575 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:40:08.331674 11364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:08.328939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:35517 (local address 127.25.62.131:43623)
0619 20:40:08.329015 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:08.329029 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:08.329110 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:08.329282 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:08.329283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:08.329286 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:08.329365 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:08.329368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.330043 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.330044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:08.330824 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:08.330825 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.331371 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.331372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:08.331454 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:08.331459 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:40:08.331531 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 20:40:08.331572 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 20:40:08.331698 11385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:08.328940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:42557 (local address 127.25.62.131:34005)
0619 20:40:08.329203 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:08.329213 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:08.329266 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:08.329394 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:08.329395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:08.329413 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:08.329484 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:08.329485 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.330104 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.330105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:08.330838 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:08.330839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.331410 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.331410 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:08.331462 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:08.331465 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:40:08.331532 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 20:40:08.331572 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":80,"thread_start_us":124,"threads_started":1}
I20260619 20:40:08.332060 11054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b5764549a0e46fda975e19c1ba9c455" candidate_uuid: "dd92a7ef70654e889444cf1267b8c565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c5df6fdf5004c6fad622537207f4344" is_pre_election: true
I20260619 20:40:08.332249 11054 raft_consensus.cc:2454] T 1b5764549a0e46fda975e19c1ba9c455 P 6c5df6fdf5004c6fad622537207f4344 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd92a7ef70654e889444cf1267b8c565 in term 0.
I20260619 20:40:08.332226 11185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b5764549a0e46fda975e19c1ba9c455" candidate_uuid: "dd92a7ef70654e889444cf1267b8c565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2d8741a94af410da9c8c37595b0617b" is_pre_election: true
I20260619 20:40:08.332384 11185 raft_consensus.cc:2454] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd92a7ef70654e889444cf1267b8c565 in term 0.
I20260619 20:40:08.332525 11249 leader_election.cc:302] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c5df6fdf5004c6fad622537207f4344, dd92a7ef70654e889444cf1267b8c565; no voters:
I20260619 20:40:08.332724 11382 raft_consensus.cc:2781] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:40:08.332775 11382 raft_consensus.cc:482] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:40:08.332823 11382 raft_consensus.cc:3037] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:08.333552 11382 raft_consensus.cc:504] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92a7ef70654e889444cf1267b8c565" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 35761 } } peers { permanent_uuid: "c2d8741a94af410da9c8c37595b0617b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 42557 } } peers { permanent_uuid: "6c5df6fdf5004c6fad622537207f4344" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 35517 } }
I20260619 20:40:08.333693 11382 leader_election.cc:288] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [CANDIDATE]: Term 1 election: Requested vote from peers c2d8741a94af410da9c8c37595b0617b (127.25.62.130:42557), 6c5df6fdf5004c6fad622537207f4344 (127.25.62.129:35517)
I20260619 20:40:08.333850 11185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b5764549a0e46fda975e19c1ba9c455" candidate_uuid: "dd92a7ef70654e889444cf1267b8c565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2d8741a94af410da9c8c37595b0617b"
I20260619 20:40:08.333845 11054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b5764549a0e46fda975e19c1ba9c455" candidate_uuid: "dd92a7ef70654e889444cf1267b8c565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c5df6fdf5004c6fad622537207f4344"
I20260619 20:40:08.333925 11185 raft_consensus.cc:3037] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:08.333945 11054 raft_consensus.cc:3037] T 1b5764549a0e46fda975e19c1ba9c455 P 6c5df6fdf5004c6fad622537207f4344 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:08.334672 11054 raft_consensus.cc:2454] T 1b5764549a0e46fda975e19c1ba9c455 P 6c5df6fdf5004c6fad622537207f4344 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd92a7ef70654e889444cf1267b8c565 in term 1.
I20260619 20:40:08.334702 11185 raft_consensus.cc:2454] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd92a7ef70654e889444cf1267b8c565 in term 1.
I20260619 20:40:08.334843 11249 leader_election.cc:302] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c5df6fdf5004c6fad622537207f4344, dd92a7ef70654e889444cf1267b8c565; no voters:
I20260619 20:40:08.334990 11382 raft_consensus.cc:2781] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:08.335163 11382 raft_consensus.cc:686] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [term 1 LEADER]: Becoming Leader. State: Replica: dd92a7ef70654e889444cf1267b8c565, State: Running, Role: LEADER
I20260619 20:40:08.335291 11382 consensus_queue.cc:237] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92a7ef70654e889444cf1267b8c565" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 35761 } } peers { permanent_uuid: "c2d8741a94af410da9c8c37595b0617b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 42557 } } peers { permanent_uuid: "6c5df6fdf5004c6fad622537207f4344" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 35517 } }
I20260619 20:40:08.336232 10912 catalog_manager.cc:5495] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd92a7ef70654e889444cf1267b8c565 (127.25.62.131). New cstate: current_term: 1 leader_uuid: "dd92a7ef70654e889444cf1267b8c565" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd92a7ef70654e889444cf1267b8c565" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 35761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2d8741a94af410da9c8c37595b0617b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 42557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c5df6fdf5004c6fad622537207f4344" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 35517 } health_report { overall_health: UNKNOWN } } }
I20260619 20:40:08.341514 11372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:08.339194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51692 (local address 127.25.62.131:35761)
0619 20:40:08.339249 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:08.339251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:08.339259 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:40:08.339298 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:08.339298 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:08.339313 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:08.339381 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:40:08.339515 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.339963 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.340809 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:08.340993 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:08.341133 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:08.341192 (+ 59us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:40:08.341281 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:40:08.341347 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:08.341380 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 20:40:08.341395 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 20:40:08.343225 11185 raft_consensus.cc:1261] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b [term 1 FOLLOWER]: Refusing update from remote peer dd92a7ef70654e889444cf1267b8c565: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:40:08.343261 11054 raft_consensus.cc:1261] T 1b5764549a0e46fda975e19c1ba9c455 P 6c5df6fdf5004c6fad622537207f4344 [term 1 FOLLOWER]: Refusing update from remote peer dd92a7ef70654e889444cf1267b8c565: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:40:08.343566 11382 consensus_queue.cc:1035] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c5df6fdf5004c6fad622537207f4344" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:40:08.343693 11386 consensus_queue.cc:1035] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2d8741a94af410da9c8c37595b0617b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 42557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:40:08.346273 11389 mvcc.cc:205] Tried to move back new op lower bound from 7298668987771265024 to 7298668987742085120. Current Snapshot: MvccSnapshot[applied={T|T < 7298668987771265024}]
I20260619 20:40:08.348256 10912 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:40:08.348362 10912 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:40:08.349098 10912 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 413cdf5e690648a785a6b6aef677ed8e: Sending DeleteTablet for 3 replicas of tablet 1b5764549a0e46fda975e19c1ba9c455
I20260619 20:40:08.349438 11296 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b5764549a0e46fda975e19c1ba9c455 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:40:08 UTC) from {username='slave'} at 127.0.0.1:51684
I20260619 20:40:08.349484 11165 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b5764549a0e46fda975e19c1ba9c455 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:40:08 UTC) from {username='slave'} at 127.0.0.1:59778
I20260619 20:40:08.349685 11034 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b5764549a0e46fda975e19c1ba9c455 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:40:08 UTC) from {username='slave'} at 127.0.0.1:40858
I20260619 20:40:08.349893 11401 tablet_replica.cc:330] T 1b5764549a0e46fda975e19c1ba9c455 P 6c5df6fdf5004c6fad622537207f4344: stopping tablet replica
I20260619 20:40:08.349967 11400 tablet_replica.cc:330] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b: stopping tablet replica
I20260619 20:40:08.350067 11400 raft_consensus.cc:2229] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:40:08.350087 11401 raft_consensus.cc:2229] T 1b5764549a0e46fda975e19c1ba9c455 P 6c5df6fdf5004c6fad622537207f4344 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:40:08.350147 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 10972
I20260619 20:40:08.350893 11400 raft_consensus.cc:2258] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:40:08.351351 11400 ts_tablet_manager.cc:1905] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:40:08.353533 11400 ts_tablet_manager.cc:1918] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:40:08.353613 11400 log.cc:1196] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/1b5764549a0e46fda975e19c1ba9c455
I20260619 20:40:08.353878 11400 ts_tablet_manager.cc:1939] T 1b5764549a0e46fda975e19c1ba9c455 P c2d8741a94af410da9c8c37595b0617b: Deleting consensus metadata
I20260619 20:40:08.354278 10899 catalog_manager.cc:4841] TS c2d8741a94af410da9c8c37595b0617b (127.25.62.130:42557): tablet 1b5764549a0e46fda975e19c1ba9c455 (table test-table [id=2924fb7a34284a2b893d4370d482d7ce]) successfully deleted
I20260619 20:40:08.354483 11399 tablet_replica.cc:330] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565: stopping tablet replica
I20260619 20:40:08.354585 11399 raft_consensus.cc:2229] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:40:08.354694 11399 raft_consensus.cc:2258] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:40:08.355157 11399 ts_tablet_manager.cc:1905] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:40:08.357167 11399 ts_tablet_manager.cc:1918] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260619 20:40:08.357197 10896 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.62.129:35517 (error 108)
I20260619 20:40:08.357241 11399 log.cc:1196] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/1b5764549a0e46fda975e19c1ba9c455
I20260619 20:40:08.357502 11399 ts_tablet_manager.cc:1939] T 1b5764549a0e46fda975e19c1ba9c455 P dd92a7ef70654e889444cf1267b8c565: Deleting consensus metadata
I20260619 20:40:08.357690 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:08.357526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:35517 (local address 127.0.0.1:40860)
0619 20:40:08.357609 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:08.357655 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.129:35517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260619 20:40:08.357801 10896 catalog_manager.cc:4584] TS 6c5df6fdf5004c6fad622537207f4344 (127.25.62.129:35517): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1b5764549a0e46fda975e19c1ba9c455: Network error: Client connection negotiation failed: client connection to 127.25.62.129:35517: connect: Connection refused (error 111)
I20260619 20:40:08.357920 10896 catalog_manager.cc:4841] TS dd92a7ef70654e889444cf1267b8c565 (127.25.62.131:35761): tablet 1b5764549a0e46fda975e19c1ba9c455 (table test-table [id=2924fb7a34284a2b893d4370d482d7ce]) successfully deleted
I20260619 20:40:08.358124 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 11103
I20260619 20:40:08.363152 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 11234
I20260619 20:40:08.367744 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 10881
2026-06-19T20:40:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (697 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:08 dist-test-slave-0t40 krb5kdc[11405](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:08 dist-test-slave-0t40 krb5kdc[11405](info): set up 2 sockets
Jun 19 20:40:08 dist-test-slave-0t40 krb5kdc[11405](info): commencing operation
krb5kdc: starting...
W20260619 20:40:09.409235 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:09 dist-test-slave-0t40 krb5kdc[11405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:40:09.525869 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:40373
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:40373
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:09.604436 11418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:09.604663 11418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:09.604697 11418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:09.605840 11418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:09.605911 11418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:09.605937 11418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:09.605964 11418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:09.605995 11418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:09.607429 11418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:40373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:40373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:09.607700 11418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:09.607919 11418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:09.612620 11423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:09.612651 11424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:09.612706 11418 server_base.cc:1034] running on GCE node
W20260619 20:40:09.612629 11426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:09.613104 11418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:09.613425 11418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:09.614588 11418 hybrid_clock.cc:648] HybridClock initialized: now 1781901609614579 us; error 38 us; skew 500 ppm
Jun 19 20:40:09 dist-test-slave-0t40 krb5kdc[11405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901609, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:09.617076 11418 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:40:09.617460 11418 webserver.cc:460] Webserver started at http://127.25.62.190:33405/ using document root <none> and password file <none>
I20260619 20:40:09.617630 11418 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:09.617667 11418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:09.617749 11418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:09.618716 11418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "e0163056a3034c1ab94c4ddd97ca6b03"
format_stamp: "Formatted at 2026-06-19 20:40:09 on dist-test-slave-0t40"
server_key: "d44fb4faac89688451eead43c4b637a4"
server_key_iv: "39c194c80c6a3135f13815bb96afcfcc"
server_key_version: "encryptionkey@0"
I20260619 20:40:09.619005 11418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "e0163056a3034c1ab94c4ddd97ca6b03"
format_stamp: "Formatted at 2026-06-19 20:40:09 on dist-test-slave-0t40"
server_key: "d44fb4faac89688451eead43c4b637a4"
server_key_iv: "39c194c80c6a3135f13815bb96afcfcc"
server_key_version: "encryptionkey@0"
I20260619 20:40:09.620445 11418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:40:09.621131 11433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:09.621277 11418 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:09.621354 11418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "e0163056a3034c1ab94c4ddd97ca6b03"
format_stamp: "Formatted at 2026-06-19 20:40:09 on dist-test-slave-0t40"
server_key: "d44fb4faac89688451eead43c4b637a4"
server_key_iv: "39c194c80c6a3135f13815bb96afcfcc"
server_key_version: "encryptionkey@0"
I20260619 20:40:09.621413 11418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:09.630002 11418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:09.631649 11418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:09.631752 11418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:09.635424 11418 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:40373
I20260619 20:40:09.635474 11485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:40373 every 8 connection(s)
I20260619 20:40:09.635777 11418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:09.636577 11486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:09.638742 11486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03: Bootstrap starting.
I20260619 20:40:09.639284 11486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:09.639555 11486 log.cc:824] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:09.640162 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 11418
I20260619 20:40:09.640275 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:09.640322 11486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03: No bootstrap required, opened a new log
I20260619 20:40:09.640493 25850 external_mini_cluster.cc:1428] Setting key fe659ed086a342ae7bc48769ee9c1d8e
I20260619 20:40:09.642132 11486 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0163056a3034c1ab94c4ddd97ca6b03" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 40373 } }
I20260619 20:40:09.642284 11486 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:09.642328 11486 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0163056a3034c1ab94c4ddd97ca6b03, State: Initialized, Role: FOLLOWER
I20260619 20:40:09.642453 11486 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0163056a3034c1ab94c4ddd97ca6b03" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 40373 } }
I20260619 20:40:09.642536 11486 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:09.642578 11486 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:09.642649 11486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:09.643536 11486 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0163056a3034c1ab94c4ddd97ca6b03" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 40373 } }
I20260619 20:40:09.643649 11486 leader_election.cc:302] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0163056a3034c1ab94c4ddd97ca6b03; no voters:
I20260619 20:40:09.643841 11486 leader_election.cc:288] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:09.643862 11491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:09.644006 11491 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03 [term 1 LEADER]: Becoming Leader. State: Replica: e0163056a3034c1ab94c4ddd97ca6b03, State: Running, Role: LEADER
I20260619 20:40:09.644084 11491 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0163056a3034c1ab94c4ddd97ca6b03" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 40373 } }
I20260619 20:40:09.644114 11486 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:09.644510 11492 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0163056a3034c1ab94c4ddd97ca6b03. Latest consensus state: current_term: 1 leader_uuid: "e0163056a3034c1ab94c4ddd97ca6b03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0163056a3034c1ab94c4ddd97ca6b03" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 40373 } } }
I20260619 20:40:09.644584 11492 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:09.644837 11496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:09.645555 11496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:09.645948 11493 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0163056a3034c1ab94c4ddd97ca6b03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0163056a3034c1ab94c4ddd97ca6b03" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 40373 } } }
I20260619 20:40:09.646035 11493 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03 [sys.catalog]: This master's current role is: LEADER
Jun 19 20:40:09 dist-test-slave-0t40 krb5kdc[11405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:40:09.649155 11496 catalog_manager.cc:1347] Generated new cluster ID: 8ba4838e9f6445fd8243d1f9fcc709fc
I20260619 20:40:09.649209 11496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:09.652010 11490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:09.640950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60868 (local address 127.25.62.190:40373)
0619 20:40:09.641161 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:09.641170 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:09.641199 (+ 29us) server_negotiation.cc:407] Connection header received
0619 20:40:09.642122 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:09.642128 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:09.642171 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:09.642301 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:09.643053 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:09.644149 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:09.645267 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:09.645898 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:09.646325 (+ 427us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:09.648392 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:09.648395 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:09.648396 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:09.648406 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:09.650373 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:09.650791 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:09.650793 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:09.650794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:09.650915 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:09.651123 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:09.651124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:09.651124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:09.651385 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:09.651527 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:09.651679 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 20:40:09.651801 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":119,"threads_started":1}
I20260619 20:40:09.669005 11496 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:09.669708 11496 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:09.674662 11496 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03: Generated new TSK 0
I20260619 20:40:09.674832 11496 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
I20260619 20:40:09.699999 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:40373
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:09.782094 11512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:09.782317 11512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:09.782354 11512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:09.783469 11512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:09.783533 11512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:09.783578 11512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:40:09.785002 11512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:40373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:09.785282 11512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:09.785508 11512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:09.790202 11520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:09.790202 11518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:09.790302 11512 server_base.cc:1034] running on GCE node
W20260619 20:40:09.790228 11517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:09.790604 11512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:09.790840 11512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:09.791990 11512 hybrid_clock.cc:648] HybridClock initialized: now 1781901609791979 us; error 25 us; skew 500 ppm
Jun 19 20:40:09 dist-test-slave-0t40 krb5kdc[11405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901609, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:09.794378 11512 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:40:09.794793 11512 webserver.cc:460] Webserver started at http://127.25.62.129:44009/ using document root <none> and password file <none>
I20260619 20:40:09.795017 11512 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:09.795083 11512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:09.795228 11512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:09.796216 11512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "64e6f9dd9eef4a29983ae048fd01046c"
format_stamp: "Formatted at 2026-06-19 20:40:09 on dist-test-slave-0t40"
server_key: "af08f16f9fd1d178c1bc7661c4497e36"
server_key_iv: "e149a129319bdfc7ede2dd05a1f4cff1"
server_key_version: "encryptionkey@0"
I20260619 20:40:09.796568 11512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "64e6f9dd9eef4a29983ae048fd01046c"
format_stamp: "Formatted at 2026-06-19 20:40:09 on dist-test-slave-0t40"
server_key: "af08f16f9fd1d178c1bc7661c4497e36"
server_key_iv: "e149a129319bdfc7ede2dd05a1f4cff1"
server_key_version: "encryptionkey@0"
I20260619 20:40:09.797935 11512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:40:09.798631 11527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:09.798799 11512 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:40:09.798890 11512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "64e6f9dd9eef4a29983ae048fd01046c"
format_stamp: "Formatted at 2026-06-19 20:40:09 on dist-test-slave-0t40"
server_key: "af08f16f9fd1d178c1bc7661c4497e36"
server_key_iv: "e149a129319bdfc7ede2dd05a1f4cff1"
server_key_version: "encryptionkey@0"
I20260619 20:40:09.798969 11512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:09.820891 11512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:09.823241 11512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:09.823431 11512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:09.823666 11512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:40:09.824007 11512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:40:09.824060 11512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:09.824102 11512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:40:09.824131 11512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:09.829824 11512 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:37583
I20260619 20:40:09.829926 11640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:37583 every 8 connection(s)
I20260619 20:40:09.830216 11512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:40:09.834939 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 11512
I20260619 20:40:09.835078 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:40:09.835304 25850 external_mini_cluster.cc:1428] Setting key 8522db45b5fbfb52eb965c4bee63541c
Jun 19 20:40:09 dist-test-slave-0t40 krb5kdc[11405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901609, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:40:09.839757 11490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:09.830907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:54263 (local address 127.25.62.190:40373)
0619 20:40:09.830959 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:09.830960 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:09.831203 (+ 243us) server_negotiation.cc:407] Connection header received
0619 20:40:09.831729 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:09.831731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:09.831743 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:09.831783 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:09.832709 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:09.833271 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:09.834304 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:09.834524 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:09.834616 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:09.836919 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:09.836921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:09.836921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:09.836929 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:09.838324 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:09.838813 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:09.838813 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:09.838814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:09.838863 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:09.839142 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:09.839143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:09.839143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:09.839237 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:09.839285 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:09.839627 (+ 342us) server_negotiation.cc:299] Negotiation successful
0619 20:40:09.839695 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 20:40:09.840022 11643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:09.830900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:40373 (local address 127.25.62.129:54263)
0619 20:40:09.831084 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:09.831118 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:09.831616 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:09.831875 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:09.831884 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:09.832427 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:09.832608 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:09.832613 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:09.833399 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:09.833400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:09.834194 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:09.834195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:09.834641 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:09.834642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:09.834799 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:09.835348 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:09.835358 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:09.836830 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:09.838404 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:09.838411 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:09.838413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:09.838735 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:09.838932 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:09.838933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:09.838934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:09.839078 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:09.839301 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:09.839307 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:09.839481 (+ 174us) client_negotiation.cc:769] Sending connection context
0619 20:40:09.839607 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 20:40:09.839779 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":98,"threads_started":1}
I20260619 20:40:09.840574 11641 heartbeater.cc:347] Connected to a master server at 127.25.62.190:40373
I20260619 20:40:09.840664 11641 heartbeater.cc:464] Registering TS with master...
I20260619 20:40:09.840852 11641 heartbeater.cc:511] Master 127.25.62.190:40373 requested a full tablet report, sending...
I20260619 20:40:09.841360 11450 ts_manager.cc:194] Registered new tserver with Master: 64e6f9dd9eef4a29983ae048fd01046c (127.25.62.129:37583)
I20260619 20:40:09.841944 11450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:54263
WARNING: no policy specified for kudu/127.25.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
I20260619 20:40:09.860457 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:40373
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:09.939134 11646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:09.939332 11646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:09.939354 11646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:09.940418 11646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:09.940449 11646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:09.940474 11646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:40:09.941847 11646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:40373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:09.942029 11646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:09.942194 11646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:09.947438 11654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:09.947438 11652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:09.947472 11651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:09.947674 11646 server_base.cc:1034] running on GCE node
I20260619 20:40:09.947836 11646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:09.948060 11646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:09.949226 11646 hybrid_clock.cc:648] HybridClock initialized: now 1781901609949214 us; error 25 us; skew 500 ppm
Jun 19 20:40:09 dist-test-slave-0t40 krb5kdc[11405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901609, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:09.951699 11646 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:40:09.952190 11646 webserver.cc:460] Webserver started at http://127.25.62.130:38443/ using document root <none> and password file <none>
I20260619 20:40:09.952427 11646 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:09.952491 11646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:09.952625 11646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:09.953815 11646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "bd000dd0318a4c0b91d211cb79e7e31b"
format_stamp: "Formatted at 2026-06-19 20:40:09 on dist-test-slave-0t40"
server_key: "d7e8f154b2872e80f88bed4e86456f75"
server_key_iv: "5deb3bcfcfd348868de8d82b2019a13d"
server_key_version: "encryptionkey@0"
I20260619 20:40:09.954166 11646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "bd000dd0318a4c0b91d211cb79e7e31b"
format_stamp: "Formatted at 2026-06-19 20:40:09 on dist-test-slave-0t40"
server_key: "d7e8f154b2872e80f88bed4e86456f75"
server_key_iv: "5deb3bcfcfd348868de8d82b2019a13d"
server_key_version: "encryptionkey@0"
I20260619 20:40:09.955477 11646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:40:09.956244 11661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:09.956388 11646 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:40:09.956477 11646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "bd000dd0318a4c0b91d211cb79e7e31b"
format_stamp: "Formatted at 2026-06-19 20:40:09 on dist-test-slave-0t40"
server_key: "d7e8f154b2872e80f88bed4e86456f75"
server_key_iv: "5deb3bcfcfd348868de8d82b2019a13d"
server_key_version: "encryptionkey@0"
I20260619 20:40:09.956530 11646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:09.979686 11646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:09.982005 11646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:09.982115 11646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:09.982295 11646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:40:09.982618 11646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:40:09.982652 11646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:09.982674 11646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:40:09.982721 11646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:09.988870 11646 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:33419
I20260619 20:40:09.988960 11774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:33419 every 8 connection(s)
I20260619 20:40:09.989279 11646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:40:09.995065 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 11646
I20260619 20:40:09.995158 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
Jun 19 20:40:09 dist-test-slave-0t40 krb5kdc[11405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901609, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:40:09.995365 25850 external_mini_cluster.cc:1428] Setting key fdc2db7e98ad04aad2a1c764ac6f455f
I20260619 20:40:09.998577 11490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:09.990014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:53023 (local address 127.25.62.190:40373)
0619 20:40:09.990099 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:09.990101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:09.990262 (+ 161us) server_negotiation.cc:407] Connection header received
0619 20:40:09.990746 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:09.990747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:09.990757 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:09.990805 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:09.991753 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:09.992260 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:09.993335 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:09.993527 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:09.993584 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:09.995872 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:09.995874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:09.995874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:09.995881 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:09.997209 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:09.997622 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:09.997624 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:09.997624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:09.997686 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:09.997935 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:09.997936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:09.997936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:09.998055 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:09.998132 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:09.998402 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 20:40:09.998486 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 20:40:09.998812 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:09.989970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:40373 (local address 127.25.62.130:53023)
0619 20:40:09.990127 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:09.990164 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:09.990658 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:09.990869 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:09.990873 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:09.991447 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:09.991642 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:09.991646 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:09.992362 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:09.992363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:09.993254 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:09.993256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:09.993626 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:09.993627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:09.993754 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:09.994223 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:09.994230 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:09.995770 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:09.997304 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:09.997310 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:09.997313 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:09.997533 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:09.997759 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:09.997760 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:09.997760 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:09.997869 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:09.998122 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:09.998128 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:09.998268 (+ 140us) client_negotiation.cc:769] Sending connection context
0619 20:40:09.998383 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 20:40:09.998539 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260619 20:40:09.999281 11775 heartbeater.cc:347] Connected to a master server at 127.25.62.190:40373
I20260619 20:40:09.999373 11775 heartbeater.cc:464] Registering TS with master...
I20260619 20:40:09.999567 11775 heartbeater.cc:511] Master 127.25.62.190:40373 requested a full tablet report, sending...
I20260619 20:40:10.001127 11450 ts_manager.cc:194] Registered new tserver with Master: bd000dd0318a4c0b91d211cb79e7e31b (127.25.62.130:33419)
I20260619 20:40:10.001573 11450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:53023
WARNING: no policy specified for kudu/127.25.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
I20260619 20:40:10.022053 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:40373
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:10.100827 11780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:10.101015 11780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:10.101055 11780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:10.102190 11780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:10.102231 11780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:10.102257 11780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:40:10.103622 11780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:40373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:10.103809 11780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:10.103977 11780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:10.109172 11786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:10.109179 11788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:10.109273 11785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:10.109380 11780 server_base.cc:1034] running on GCE node
I20260619 20:40:10.109616 11780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:10.109872 11780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:10.111081 11780 hybrid_clock.cc:648] HybridClock initialized: now 1781901610111055 us; error 38 us; skew 500 ppm
Jun 19 20:40:10 dist-test-slave-0t40 krb5kdc[11405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901610, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:10.113696 11780 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:40:10.114168 11780 webserver.cc:460] Webserver started at http://127.25.62.131:32949/ using document root <none> and password file <none>
I20260619 20:40:10.114400 11780 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:10.114476 11780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:10.114627 11780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:10.115661 11780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "1bcb6572e18c48de902d12aadad45be1"
format_stamp: "Formatted at 2026-06-19 20:40:10 on dist-test-slave-0t40"
server_key: "7044d88a60040e9f723e531169bf79fa"
server_key_iv: "996f69bd650ca7c410f7c06ac6417270"
server_key_version: "encryptionkey@0"
I20260619 20:40:10.116017 11780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "1bcb6572e18c48de902d12aadad45be1"
format_stamp: "Formatted at 2026-06-19 20:40:10 on dist-test-slave-0t40"
server_key: "7044d88a60040e9f723e531169bf79fa"
server_key_iv: "996f69bd650ca7c410f7c06ac6417270"
server_key_version: "encryptionkey@0"
I20260619 20:40:10.117353 11780 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:40:10.118186 11795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:10.118409 11780 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:40:10.118522 11780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "1bcb6572e18c48de902d12aadad45be1"
format_stamp: "Formatted at 2026-06-19 20:40:10 on dist-test-slave-0t40"
server_key: "7044d88a60040e9f723e531169bf79fa"
server_key_iv: "996f69bd650ca7c410f7c06ac6417270"
server_key_version: "encryptionkey@0"
I20260619 20:40:10.118597 11780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:10.126227 11780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:10.128425 11780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:10.128580 11780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:10.128805 11780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:40:10.129149 11780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:40:10.129204 11780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:10.129238 11780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:40:10.129266 11780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:10.135257 11780 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:44329
I20260619 20:40:10.135317 11908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:44329 every 8 connection(s)
I20260619 20:40:10.135627 11780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:40:10.136531 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 11780
I20260619 20:40:10.136636 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:40:10.136837 25850 external_mini_cluster.cc:1428] Setting key 5a6ef2a04a2e24b55814793b439553d0
Jun 19 20:40:10 dist-test-slave-0t40 krb5kdc[11405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901610, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:40:10.144740 11490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.136353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:41591 (local address 127.25.62.190:40373)
0619 20:40:10.136434 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:10.136436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:10.136645 (+ 209us) server_negotiation.cc:407] Connection header received
0619 20:40:10.137322 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:10.137323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:10.137334 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:10.137377 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:10.138281 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.138812 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.139811 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.140008 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.140092 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.142121 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:10.142122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:10.142123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:10.142130 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:10.143384 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:10.143825 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:10.143826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:10.143827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:10.143885 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:10.144103 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:10.144104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:10.144104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:10.144208 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:10.144263 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:10.144571 (+ 308us) server_negotiation.cc:299] Negotiation successful
0619 20:40:10.144659 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 20:40:10.144915 11911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.136313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:40373 (local address 127.25.62.131:41591)
0619 20:40:10.136501 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:10.136548 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:10.137223 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:10.137442 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:10.137446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:10.138000 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:10.138183 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.138188 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.138914 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.138916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.139717 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.139719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.140101 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.140102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.140228 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.140685 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:10.140692 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:10.142020 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:10.143499 (+ 1479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:10.143505 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:10.143507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:10.143734 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:10.143949 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:10.143950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:10.143950 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:10.144046 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:10.144261 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:10.144265 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:10.144443 (+ 178us) client_negotiation.cc:769] Sending connection context
0619 20:40:10.144527 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 20:40:10.144706 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":89,"threads_started":1}
I20260619 20:40:10.145421 11909 heartbeater.cc:347] Connected to a master server at 127.25.62.190:40373
I20260619 20:40:10.145516 11909 heartbeater.cc:464] Registering TS with master...
I20260619 20:40:10.145689 11909 heartbeater.cc:511] Master 127.25.62.190:40373 requested a full tablet report, sending...
I20260619 20:40:10.146068 11450 ts_manager.cc:194] Registered new tserver with Master: 1bcb6572e18c48de902d12aadad45be1 (127.25.62.131:44329)
I20260619 20:40:10.146442 11450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:41591
I20260619 20:40:10.148643 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:40:10.157251 11490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.150216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60872 (local address 127.25.62.190:40373)
0619 20:40:10.150297 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:10.150299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:10.150392 (+ 93us) server_negotiation.cc:407] Connection header received
0619 20:40:10.150447 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:10.150448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:10.150456 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:10.150501 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:10.151076 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.151552 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.152322 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.152600 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.152674 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.154344 (+ 1670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:10.154346 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:10.154346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:10.154354 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:10.155968 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:10.156373 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:10.156373 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:10.156374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:10.156422 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:10.156762 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:10.156763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:10.156763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:10.156888 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:10.156970 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:10.157144 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 20:40:10.157192 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 20:40:10.159058 11450 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60872:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:40:10.160683 11450 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:40:10.167229 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.162856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59454 (local address 127.25.62.129:37583)
0619 20:40:10.163074 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:10.163080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:10.163091 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:40:10.163132 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:10.163135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:10.163151 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:10.163236 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:10.163476 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.164613 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.166172 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.166836 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.166900 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.166947 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:10.167101 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 20:40:10.167142 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":108,"threads_started":1}
I20260619 20:40:10.167258 11920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.163072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:44329 (local address 127.0.0.1:46326)
0619 20:40:10.163248 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:10.163259 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:10.163308 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:10.163865 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:10.163866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:10.163868 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:10.163906 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.163907 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.165017 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.165018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.165964 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.165966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.166905 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.166906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.166988 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.167003 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 20:40:10.167076 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 20:40:10.167143 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":25,"thread_start_us":61,"threads_started":1}
I20260619 20:40:10.167258 11917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.162718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:37583 (local address 127.0.0.1:59454)
0619 20:40:10.162851 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:10.162894 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:10.162964 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:10.163342 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:10.163345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:10.163347 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:10.163401 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.163403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.164707 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.164707 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.166082 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.166083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.166905 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.166906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.166997 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.167003 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 20:40:10.167076 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 20:40:10.167143 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":100,"thread_start_us":60,"threads_started":1}
I20260619 20:40:10.167477 11922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.163165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46326 (local address 127.25.62.131:44329)
0619 20:40:10.163707 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:10.163712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:10.163720 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:40:10.163760 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:10.163763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:10.163777 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:10.163855 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:10.163966 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.164916 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.166047 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.166796 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.166874 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.166935 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:10.167364 (+ 429us) server_negotiation.cc:299] Negotiation successful
0619 20:40:10.167402 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":87,"threads_started":1}
I20260619 20:40:10.168156 11919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.162943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:33419 (local address 127.0.0.1:56572)
0619 20:40:10.163179 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:10.163188 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:10.163228 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:10.163567 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:10.163568 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:10.163570 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:10.163616 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.163618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.164574 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.164575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.166080 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.166083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.167925 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.167926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.167990 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.167994 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:40:10.168046 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 20:40:10.168087 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":72,"thread_start_us":38,"threads_started":1}
I20260619 20:40:10.168174 11921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.163068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56572 (local address 127.25.62.130:33419)
0619 20:40:10.163375 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:10.163380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:10.163389 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:40:10.163471 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:10.163473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:10.163489 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:10.163569 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:10.163700 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.164475 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.166988 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.167771 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.167835 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.167893 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:10.168061 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 20:40:10.168098 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":94,"threads_started":1}
I20260619 20:40:10.168097 11843 tablet_service.cc:1467] Processing CreateTablet for tablet 8de2ee6ca6a343dabd3a4282c3c43444 (DEFAULT_TABLE table=test-table [id=5e17158f22ca4ff4b7e28207b58e2097]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:40:10.168097 11575 tablet_service.cc:1467] Processing CreateTablet for tablet 8de2ee6ca6a343dabd3a4282c3c43444 (DEFAULT_TABLE table=test-table [id=5e17158f22ca4ff4b7e28207b58e2097]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:40:10.168432 11575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8de2ee6ca6a343dabd3a4282c3c43444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:10.168432 11843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8de2ee6ca6a343dabd3a4282c3c43444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:10.168756 11709 tablet_service.cc:1467] Processing CreateTablet for tablet 8de2ee6ca6a343dabd3a4282c3c43444 (DEFAULT_TABLE table=test-table [id=5e17158f22ca4ff4b7e28207b58e2097]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:40:10.168987 11709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8de2ee6ca6a343dabd3a4282c3c43444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:10.171700 11923 tablet_bootstrap.cc:492] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b: Bootstrap starting.
I20260619 20:40:10.171828 11924 tablet_bootstrap.cc:492] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1: Bootstrap starting.
I20260619 20:40:10.172101 11925 tablet_bootstrap.cc:492] T 8de2ee6ca6a343dabd3a4282c3c43444 P 64e6f9dd9eef4a29983ae048fd01046c: Bootstrap starting.
I20260619 20:40:10.172348 11923 tablet_bootstrap.cc:654] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:10.172510 11924 tablet_bootstrap.cc:654] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:10.172711 11925 tablet_bootstrap.cc:654] T 8de2ee6ca6a343dabd3a4282c3c43444 P 64e6f9dd9eef4a29983ae048fd01046c: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:10.172950 11923 log.cc:824] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:10.173148 11924 log.cc:824] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:10.173348 11925 log.cc:824] T 8de2ee6ca6a343dabd3a4282c3c43444 P 64e6f9dd9eef4a29983ae048fd01046c: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:10.173810 11923 tablet_bootstrap.cc:492] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b: No bootstrap required, opened a new log
I20260619 20:40:10.173869 11924 tablet_bootstrap.cc:492] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1: No bootstrap required, opened a new log
I20260619 20:40:10.173882 11923 ts_tablet_manager.cc:1397] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:40:10.173920 11924 ts_tablet_manager.cc:1397] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:40:10.174060 11925 tablet_bootstrap.cc:492] T 8de2ee6ca6a343dabd3a4282c3c43444 P 64e6f9dd9eef4a29983ae048fd01046c: No bootstrap required, opened a new log
I20260619 20:40:10.174108 11925 ts_tablet_manager.cc:1397] T 8de2ee6ca6a343dabd3a4282c3c43444 P 64e6f9dd9eef4a29983ae048fd01046c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:40:10.175537 11924 raft_consensus.cc:348] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e6f9dd9eef4a29983ae048fd01046c" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 37583 } } peers { permanent_uuid: "bd000dd0318a4c0b91d211cb79e7e31b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33419 } } peers { permanent_uuid: "1bcb6572e18c48de902d12aadad45be1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44329 } }
I20260619 20:40:10.175665 11924 raft_consensus.cc:374] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:10.175685 11924 raft_consensus.cc:729] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bcb6572e18c48de902d12aadad45be1, State: Initialized, Role: FOLLOWER
I20260619 20:40:10.175812 11924 consensus_queue.cc:260] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e6f9dd9eef4a29983ae048fd01046c" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 37583 } } peers { permanent_uuid: "bd000dd0318a4c0b91d211cb79e7e31b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33419 } } peers { permanent_uuid: "1bcb6572e18c48de902d12aadad45be1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44329 } }
I20260619 20:40:10.175840 11923 raft_consensus.cc:348] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e6f9dd9eef4a29983ae048fd01046c" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 37583 } } peers { permanent_uuid: "bd000dd0318a4c0b91d211cb79e7e31b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33419 } } peers { permanent_uuid: "1bcb6572e18c48de902d12aadad45be1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44329 } }
I20260619 20:40:10.175993 11923 raft_consensus.cc:374] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:10.176029 11923 raft_consensus.cc:729] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd000dd0318a4c0b91d211cb79e7e31b, State: Initialized, Role: FOLLOWER
I20260619 20:40:10.176017 11925 raft_consensus.cc:348] T 8de2ee6ca6a343dabd3a4282c3c43444 P 64e6f9dd9eef4a29983ae048fd01046c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e6f9dd9eef4a29983ae048fd01046c" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 37583 } } peers { permanent_uuid: "bd000dd0318a4c0b91d211cb79e7e31b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33419 } } peers { permanent_uuid: "1bcb6572e18c48de902d12aadad45be1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44329 } }
I20260619 20:40:10.176155 11925 raft_consensus.cc:374] T 8de2ee6ca6a343dabd3a4282c3c43444 P 64e6f9dd9eef4a29983ae048fd01046c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:10.176142 11923 consensus_queue.cc:260] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e6f9dd9eef4a29983ae048fd01046c" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 37583 } } peers { permanent_uuid: "bd000dd0318a4c0b91d211cb79e7e31b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33419 } } peers { permanent_uuid: "1bcb6572e18c48de902d12aadad45be1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44329 } }
I20260619 20:40:10.176187 11925 raft_consensus.cc:729] T 8de2ee6ca6a343dabd3a4282c3c43444 P 64e6f9dd9eef4a29983ae048fd01046c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64e6f9dd9eef4a29983ae048fd01046c, State: Initialized, Role: FOLLOWER
I20260619 20:40:10.176252 11909 heartbeater.cc:503] Master 127.25.62.190:40373 was elected leader, sending a full tablet report...
I20260619 20:40:10.176290 11924 ts_tablet_manager.cc:1428] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:40:10.176304 11925 consensus_queue.cc:260] T 8de2ee6ca6a343dabd3a4282c3c43444 P 64e6f9dd9eef4a29983ae048fd01046c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e6f9dd9eef4a29983ae048fd01046c" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 37583 } } peers { permanent_uuid: "bd000dd0318a4c0b91d211cb79e7e31b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33419 } } peers { permanent_uuid: "1bcb6572e18c48de902d12aadad45be1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44329 } }
I20260619 20:40:10.176409 11923 ts_tablet_manager.cc:1428] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:40:10.176528 11775 heartbeater.cc:503] Master 127.25.62.190:40373 was elected leader, sending a full tablet report...
I20260619 20:40:10.176613 11641 heartbeater.cc:503] Master 127.25.62.190:40373 was elected leader, sending a full tablet report...
I20260619 20:40:10.176939 11925 ts_tablet_manager.cc:1428] T 8de2ee6ca6a343dabd3a4282c3c43444 P 64e6f9dd9eef4a29983ae048fd01046c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:40:10.211843 11929 raft_consensus.cc:482] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:40:10.211961 11929 raft_consensus.cc:504] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e6f9dd9eef4a29983ae048fd01046c" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 37583 } } peers { permanent_uuid: "bd000dd0318a4c0b91d211cb79e7e31b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33419 } } peers { permanent_uuid: "1bcb6572e18c48de902d12aadad45be1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44329 } }
I20260619 20:40:10.212249 11929 leader_election.cc:288] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64e6f9dd9eef4a29983ae048fd01046c (127.25.62.129:37583), bd000dd0318a4c0b91d211cb79e7e31b (127.25.62.130:33419)
I20260619 20:40:10.215395 11921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.212712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:34993 (local address 127.25.62.130:33419)
0619 20:40:10.212764 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:10.212766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:10.212877 (+ 111us) server_negotiation.cc:407] Connection header received
0619 20:40:10.212906 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:10.212908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:10.212918 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:10.212973 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:10.213160 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.213662 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.214634 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.215015 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.215123 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.215133 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:10.215267 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 20:40:10.215318 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 20:40:10.215405 11911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.212480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:37583 (local address 127.25.62.131:39661)
0619 20:40:10.212541 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:10.212554 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:10.212600 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:10.212797 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:10.212797 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:10.212800 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:10.212904 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.212905 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.213650 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.213651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.214472 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.214474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.215101 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.215102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.215192 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.215209 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 20:40:10.215287 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 20:40:10.215327 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":13}
I20260619 20:40:10.215423 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.212565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:39661 (local address 127.25.62.129:37583)
0619 20:40:10.212630 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:10.212633 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:10.212644 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:40:10.212675 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:10.212676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:10.212687 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:10.212756 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:10.212992 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.213564 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.214571 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.215011 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.215092 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.215102 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:10.215317 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 20:40:10.215362 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 20:40:10.215413 11932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.212600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:33419 (local address 127.25.62.131:34993)
0619 20:40:10.212777 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:10.212788 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:10.212863 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:10.212982 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:10.212983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:10.212985 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:10.213075 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.213077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.213751 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.213752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.214542 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.214544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.215126 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.215127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.215192 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.215196 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:40:10.215240 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 20:40:10.215300 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":121,"threads_started":1}
I20260619 20:40:10.215745 11729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8de2ee6ca6a343dabd3a4282c3c43444" candidate_uuid: "1bcb6572e18c48de902d12aadad45be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd000dd0318a4c0b91d211cb79e7e31b" is_pre_election: true
I20260619 20:40:10.215780 11595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8de2ee6ca6a343dabd3a4282c3c43444" candidate_uuid: "1bcb6572e18c48de902d12aadad45be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64e6f9dd9eef4a29983ae048fd01046c" is_pre_election: true
I20260619 20:40:10.215930 11729 raft_consensus.cc:2454] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bcb6572e18c48de902d12aadad45be1 in term 0.
I20260619 20:40:10.215930 11595 raft_consensus.cc:2454] T 8de2ee6ca6a343dabd3a4282c3c43444 P 64e6f9dd9eef4a29983ae048fd01046c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bcb6572e18c48de902d12aadad45be1 in term 0.
I20260619 20:40:10.216265 11799 leader_election.cc:302] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcb6572e18c48de902d12aadad45be1, 64e6f9dd9eef4a29983ae048fd01046c; no voters:
I20260619 20:40:10.216403 11929 raft_consensus.cc:2781] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:40:10.216493 11929 raft_consensus.cc:482] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:40:10.216528 11929 raft_consensus.cc:3037] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:10.217475 11929 raft_consensus.cc:504] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e6f9dd9eef4a29983ae048fd01046c" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 37583 } } peers { permanent_uuid: "bd000dd0318a4c0b91d211cb79e7e31b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33419 } } peers { permanent_uuid: "1bcb6572e18c48de902d12aadad45be1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44329 } }
I20260619 20:40:10.217651 11929 leader_election.cc:288] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [CANDIDATE]: Term 1 election: Requested vote from peers 64e6f9dd9eef4a29983ae048fd01046c (127.25.62.129:37583), bd000dd0318a4c0b91d211cb79e7e31b (127.25.62.130:33419)
I20260619 20:40:10.217931 11595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8de2ee6ca6a343dabd3a4282c3c43444" candidate_uuid: "1bcb6572e18c48de902d12aadad45be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64e6f9dd9eef4a29983ae048fd01046c"
I20260619 20:40:10.217931 11729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8de2ee6ca6a343dabd3a4282c3c43444" candidate_uuid: "1bcb6572e18c48de902d12aadad45be1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd000dd0318a4c0b91d211cb79e7e31b"
I20260619 20:40:10.218026 11595 raft_consensus.cc:3037] T 8de2ee6ca6a343dabd3a4282c3c43444 P 64e6f9dd9eef4a29983ae048fd01046c [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:10.218026 11729 raft_consensus.cc:3037] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:10.218941 11595 raft_consensus.cc:2454] T 8de2ee6ca6a343dabd3a4282c3c43444 P 64e6f9dd9eef4a29983ae048fd01046c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcb6572e18c48de902d12aadad45be1 in term 1.
I20260619 20:40:10.218941 11729 raft_consensus.cc:2454] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bcb6572e18c48de902d12aadad45be1 in term 1.
I20260619 20:40:10.219139 11799 leader_election.cc:302] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bcb6572e18c48de902d12aadad45be1, 64e6f9dd9eef4a29983ae048fd01046c; no voters:
I20260619 20:40:10.219265 11929 raft_consensus.cc:2781] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:10.219412 11929 raft_consensus.cc:686] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [term 1 LEADER]: Becoming Leader. State: Replica: 1bcb6572e18c48de902d12aadad45be1, State: Running, Role: LEADER
I20260619 20:40:10.219512 11929 consensus_queue.cc:237] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e6f9dd9eef4a29983ae048fd01046c" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 37583 } } peers { permanent_uuid: "bd000dd0318a4c0b91d211cb79e7e31b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33419 } } peers { permanent_uuid: "1bcb6572e18c48de902d12aadad45be1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44329 } }
I20260619 20:40:10.220502 11449 catalog_manager.cc:5495] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bcb6572e18c48de902d12aadad45be1 (127.25.62.131). New cstate: current_term: 1 leader_uuid: "1bcb6572e18c48de902d12aadad45be1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64e6f9dd9eef4a29983ae048fd01046c" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 37583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd000dd0318a4c0b91d211cb79e7e31b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bcb6572e18c48de902d12aadad45be1" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 44329 } health_report { overall_health: HEALTHY } } }
W20260619 20:40:10.239849 11776 tablet.cc:2369] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:40:10.240211 11922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.237791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46342 (local address 127.25.62.131:44329)
0619 20:40:10.237861 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:10.237863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:10.237875 (+ 12us) server_negotiation.cc:407] Connection header received
0619 20:40:10.237930 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:10.237931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:10.237949 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:10.238031 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:40:10.238175 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.238643 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.239421 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.239612 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.239684 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.239836 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:40:10.239959 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:40:10.240060 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:10.240110 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 20:40:10.240121 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 20:40:10.241961 11729 raft_consensus.cc:1261] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b [term 1 FOLLOWER]: Refusing update from remote peer 1bcb6572e18c48de902d12aadad45be1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:40:10.241974 11595 raft_consensus.cc:1261] T 8de2ee6ca6a343dabd3a4282c3c43444 P 64e6f9dd9eef4a29983ae048fd01046c [term 1 FOLLOWER]: Refusing update from remote peer 1bcb6572e18c48de902d12aadad45be1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:40:10.242336 11933 consensus_queue.cc:1035] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd000dd0318a4c0b91d211cb79e7e31b" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 33419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:40:10.242455 11929 consensus_queue.cc:1035] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64e6f9dd9eef4a29983ae048fd01046c" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 37583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:40:10.244815 11936 mvcc.cc:205] Tried to move back new op lower bound from 7298668995548413952 to 7298668995459821568. Current Snapshot: MvccSnapshot[applied={T|T < 7298668995548413952}]
I20260619 20:40:10.246090 11938 mvcc.cc:205] Tried to move back new op lower bound from 7298668995548413952 to 7298668995459821568. Current Snapshot: MvccSnapshot[applied={T|T < 7298668995548413952 or (T in {7298668995548413952})}]
I20260619 20:40:10.247462 11448 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:40:10.247562 11448 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:40:10.248430 11448 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0163056a3034c1ab94c4ddd97ca6b03: Sending DeleteTablet for 3 replicas of tablet 8de2ee6ca6a343dabd3a4282c3c43444
I20260619 20:40:10.248822 11575 tablet_service.cc:1514] Processing DeleteTablet for tablet 8de2ee6ca6a343dabd3a4282c3c43444 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:40:10 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:59454
I20260619 20:40:10.249074 11709 tablet_service.cc:1514] Processing DeleteTablet for tablet 8de2ee6ca6a343dabd3a4282c3c43444 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:40:10 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:56572
I20260619 20:40:10.249125 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 11512
I20260619 20:40:10.249132 11843 tablet_service.cc:1514] Processing DeleteTablet for tablet 8de2ee6ca6a343dabd3a4282c3c43444 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:40:10 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:46326
I20260619 20:40:10.249312 11946 tablet_replica.cc:330] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b: stopping tablet replica
I20260619 20:40:10.249399 11946 raft_consensus.cc:2229] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:40:10.249491 11946 raft_consensus.cc:2258] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:40:10.249850 11947 tablet_replica.cc:330] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1: stopping tablet replica
I20260619 20:40:10.249939 11947 raft_consensus.cc:2229] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:40:10.250025 11947 raft_consensus.cc:2258] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:40:10.250468 11947 ts_tablet_manager.cc:1905] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:40:10.252442 11947 ts_tablet_manager.cc:1918] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:40:10.252511 11947 log.cc:1196] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/8de2ee6ca6a343dabd3a4282c3c43444
I20260619 20:40:10.252795 11947 ts_tablet_manager.cc:1939] T 8de2ee6ca6a343dabd3a4282c3c43444 P 1bcb6572e18c48de902d12aadad45be1: Deleting consensus metadata
I20260619 20:40:10.253189 11437 catalog_manager.cc:4841] TS 1bcb6572e18c48de902d12aadad45be1 (127.25.62.131:44329): tablet 8de2ee6ca6a343dabd3a4282c3c43444 (table test-table [id=5e17158f22ca4ff4b7e28207b58e2097]) successfully deleted
I20260619 20:40:10.253614 11946 ts_tablet_manager.cc:1905] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:40:10.255499 11946 ts_tablet_manager.cc:1918] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:40:10.255576 11946 log.cc:1196] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/8de2ee6ca6a343dabd3a4282c3c43444
I20260619 20:40:10.255848 11946 ts_tablet_manager.cc:1939] T 8de2ee6ca6a343dabd3a4282c3c43444 P bd000dd0318a4c0b91d211cb79e7e31b: Deleting consensus metadata
I20260619 20:40:10.256224 11437 catalog_manager.cc:4841] TS bd000dd0318a4c0b91d211cb79e7e31b (127.25.62.130:33419): tablet 8de2ee6ca6a343dabd3a4282c3c43444 (table test-table [id=5e17158f22ca4ff4b7e28207b58e2097]) successfully deleted
W20260619 20:40:10.257413 11437 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.62.129:37583 (error 108)
I20260619 20:40:10.257929 11917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.257810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:37583 (local address 127.0.0.1:59460)
0619 20:40:10.257859 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:10.257894 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.129:37583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 20:40:10.257998 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 11646
W20260619 20:40:10.258054 11437 catalog_manager.cc:4584] TS 64e6f9dd9eef4a29983ae048fd01046c (127.25.62.129:37583): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8de2ee6ca6a343dabd3a4282c3c43444: Network error: Client connection negotiation failed: client connection to 127.25.62.129:37583: connect: Connection refused (error 111)
I20260619 20:40:10.263108 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 11780
I20260619 20:40:10.267557 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 11418
2026-06-19T20:40:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1899 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T20:40:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:10Z Disabled control of system clock
I20260619 20:40:10.293761 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:36339
--webserver_interface=127.25.62.190
--webserver_port=0
--builtin_ntp_servers=127.25.62.148:43539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:36339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:40:10.374496 11952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:10.374764 11952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:10.374800 11952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:10.376040 11952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:10.376088 11952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:10.376103 11952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:10.376117 11952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:10.376132 11952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:10.377636 11952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:43539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:36339
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:36339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:10.377894 11952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:10.378193 11952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:10.383051 11957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:10.383117 11960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:10.383172 11952 server_base.cc:1034] running on GCE node
W20260619 20:40:10.383102 11958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:10.383495 11952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:10.383833 11952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:10.385020 11952 hybrid_clock.cc:648] HybridClock initialized: now 1781901610385006 us; error 41 us; skew 500 ppm
I20260619 20:40:10.386449 11952 webserver.cc:460] Webserver started at http://127.25.62.190:43439/ using document root <none> and password file <none>
I20260619 20:40:10.386723 11952 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:10.386797 11952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:10.386960 11952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:10.388001 11952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "87001c14e0e148de871079373b9074e3"
format_stamp: "Formatted at 2026-06-19 20:40:10 on dist-test-slave-0t40"
server_key: "e1818a9a664b33a046718c94fa1d50bc"
server_key_iv: "1eb74b60d800cdce1cf76ef215f22ecc"
server_key_version: "encryptionkey@0"
I20260619 20:40:10.388382 11952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "87001c14e0e148de871079373b9074e3"
format_stamp: "Formatted at 2026-06-19 20:40:10 on dist-test-slave-0t40"
server_key: "e1818a9a664b33a046718c94fa1d50bc"
server_key_iv: "1eb74b60d800cdce1cf76ef215f22ecc"
server_key_version: "encryptionkey@0"
I20260619 20:40:10.389845 11952 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 20:40:10.390647 11966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:10.390834 11952 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:10.390942 11952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "87001c14e0e148de871079373b9074e3"
format_stamp: "Formatted at 2026-06-19 20:40:10 on dist-test-slave-0t40"
server_key: "e1818a9a664b33a046718c94fa1d50bc"
server_key_iv: "1eb74b60d800cdce1cf76ef215f22ecc"
server_key_version: "encryptionkey@0"
I20260619 20:40:10.391022 11952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:10.406736 11952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:10.407045 11952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:10.407199 11952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:10.411311 11952 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:36339
I20260619 20:40:10.411368 12018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:36339 every 8 connection(s)
I20260619 20:40:10.411751 11952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:10.412844 12019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:10.415131 12019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3: Bootstrap starting.
I20260619 20:40:10.415704 12019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:10.416007 12019 log.cc:824] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:10.416676 12019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3: No bootstrap required, opened a new log
I20260619 20:40:10.418022 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 11952
I20260619 20:40:10.418135 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:10.418316 25850 external_mini_cluster.cc:1428] Setting key cbaba0b04c61198a6c5ba6bed0377a96
I20260619 20:40:10.418605 12019 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87001c14e0e148de871079373b9074e3" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 36339 } }
I20260619 20:40:10.418815 12019 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:10.418907 12019 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87001c14e0e148de871079373b9074e3, State: Initialized, Role: FOLLOWER
I20260619 20:40:10.419067 12019 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87001c14e0e148de871079373b9074e3" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 36339 } }
I20260619 20:40:10.419165 12019 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:10.419198 12019 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:10.419250 12019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:10.419965 12019 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87001c14e0e148de871079373b9074e3" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 36339 } }
I20260619 20:40:10.420085 12019 leader_election.cc:302] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87001c14e0e148de871079373b9074e3; no voters:
I20260619 20:40:10.420261 12019 leader_election.cc:288] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:10.420341 12024 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:10.420495 12019 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:10.420508 12024 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3 [term 1 LEADER]: Becoming Leader. State: Replica: 87001c14e0e148de871079373b9074e3, State: Running, Role: LEADER
I20260619 20:40:10.420644 12024 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87001c14e0e148de871079373b9074e3" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 36339 } }
I20260619 20:40:10.421048 12024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87001c14e0e148de871079373b9074e3. Latest consensus state: current_term: 1 leader_uuid: "87001c14e0e148de871079373b9074e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87001c14e0e148de871079373b9074e3" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 36339 } } }
I20260619 20:40:10.421139 12024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:10.421298 12025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87001c14e0e148de871079373b9074e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87001c14e0e148de871079373b9074e3" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 36339 } } }
I20260619 20:40:10.421363 12025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3 [sys.catalog]: This master's current role is: LEADER
W20260619 20:40:10.422353 12039 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:40:10.422410 12039 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:40:10.422505 12033 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:10.422722 12033 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:10.424314 12033 catalog_manager.cc:1347] Generated new cluster ID: b5fc8df5500c47e28fcb0fd2df5630a5
I20260619 20:40:10.424378 12033 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:10.424834 12023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.418861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54202 (local address 127.25.62.190:36339)
0619 20:40:10.419185 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:10.419194 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:10.419225 (+ 31us) server_negotiation.cc:407] Connection header received
0619 20:40:10.419338 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:10.419345 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:10.419426 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:10.419567 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:10.419769 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.421673 (+ 1904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.422647 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.423230 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.423366 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.424103 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:10.424106 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:10.424107 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:10.424308 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:10.424351 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:10.424367 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:10.424434 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:10.424544 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 20:40:10.424629 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":173,"threads_started":1}
I20260619 20:40:10.435731 12033 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:10.436228 12033 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:10.440487 12033 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3: Generated new TSK 0
I20260619 20:40:10.440668 12033 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:10.444489 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:36339
--builtin_ntp_servers=127.25.62.148:43539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:40:10.531599 12043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:10.531788 12043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:10.531809 12043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:10.532933 12043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:10.532971 12043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:10.532995 12043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:40:10.534333 12043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:43539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:36339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:10.534596 12043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:10.534891 12043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:10.539844 12048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:10.540015 12049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:10.539855 12051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:10.540493 12043 server_base.cc:1034] running on GCE node
I20260619 20:40:10.540694 12043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:10.540930 12043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:10.542088 12043 hybrid_clock.cc:648] HybridClock initialized: now 1781901610542084 us; error 44 us; skew 500 ppm
I20260619 20:40:10.543329 12043 webserver.cc:460] Webserver started at http://127.25.62.129:42299/ using document root <none> and password file <none>
I20260619 20:40:10.543577 12043 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:10.543649 12043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:10.543807 12043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:10.544847 12043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "984cbec8aeff4c2c8024852e5835bacb"
format_stamp: "Formatted at 2026-06-19 20:40:10 on dist-test-slave-0t40"
server_key: "90e515275b24d37376e2af18092f98b4"
server_key_iv: "812d28e0ad9ee5de6152a137ace70c52"
server_key_version: "encryptionkey@0"
I20260619 20:40:10.545218 12043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "984cbec8aeff4c2c8024852e5835bacb"
format_stamp: "Formatted at 2026-06-19 20:40:10 on dist-test-slave-0t40"
server_key: "90e515275b24d37376e2af18092f98b4"
server_key_iv: "812d28e0ad9ee5de6152a137ace70c52"
server_key_version: "encryptionkey@0"
I20260619 20:40:10.546561 12043 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:40:10.547348 12057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:10.547518 12043 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:40:10.547613 12043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "984cbec8aeff4c2c8024852e5835bacb"
format_stamp: "Formatted at 2026-06-19 20:40:10 on dist-test-slave-0t40"
server_key: "90e515275b24d37376e2af18092f98b4"
server_key_iv: "812d28e0ad9ee5de6152a137ace70c52"
server_key_version: "encryptionkey@0"
I20260619 20:40:10.547693 12043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:10.558454 12043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:10.558734 12043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:10.558851 12043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:10.559024 12043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:40:10.559403 12043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:40:10.559449 12043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:10.559513 12043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:40:10.559566 12043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:10.565981 12043 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:33145
I20260619 20:40:10.566056 12170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:33145 every 8 connection(s)
I20260619 20:40:10.566435 12043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:40:10.569818 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 12043
I20260619 20:40:10.569906 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:40:10.570092 25850 external_mini_cluster.cc:1428] Setting key bacf3f0d710ef9595cc885322305b29e
I20260619 20:40:10.570999 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:36339
--builtin_ntp_servers=127.25.62.148:43539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 20:40:10.572176 12023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.567177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:34385 (local address 127.25.62.190:36339)
0619 20:40:10.567252 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:10.567254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:10.567479 (+ 225us) server_negotiation.cc:407] Connection header received
0619 20:40:10.568034 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:10.568036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:10.568054 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:10.568125 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:10.568820 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.569405 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.570572 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.570776 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.570872 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.571844 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:10.571846 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:10.571846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:10.571854 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:10.571875 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:10.571879 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:10.571952 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:10.572058 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 20:40:10.572092 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 20:40:10.572434 12173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.567172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:36339 (local address 127.25.62.129:34385)
0619 20:40:10.567354 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:10.567388 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:10.567919 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:10.568198 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:10.568213 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:10.568221 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:10.568715 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.568720 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.569523 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.569525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.570461 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.570464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.570897 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.570898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.571030 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.571687 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:40:10.571687 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:10.571719 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:40:10.571741 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:10.571955 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:10.571962 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:10.571965 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:40:10.572061 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 20:40:10.572167 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":73,"threads_started":1}
I20260619 20:40:10.573096 12171 heartbeater.cc:347] Connected to a master server at 127.25.62.190:36339
I20260619 20:40:10.573213 12171 heartbeater.cc:464] Registering TS with master...
I20260619 20:40:10.573395 12171 heartbeater.cc:511] Master 127.25.62.190:36339 requested a full tablet report, sending...
I20260619 20:40:10.573961 11983 ts_manager.cc:194] Registered new tserver with Master: 984cbec8aeff4c2c8024852e5835bacb (127.25.62.129:33145)
I20260619 20:40:10.574594 11983 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.62.129:34385
W20260619 20:40:10.651957 12174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:10.652145 12174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:10.652166 12174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:10.653326 12174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:10.653363 12174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:10.653388 12174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:40:10.654724 12174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:43539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:36339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:10.654963 12174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:10.655246 12174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:10.660418 12180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:10.660449 12179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:10.660588 12182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:10.660634 12174 server_base.cc:1034] running on GCE node
I20260619 20:40:10.660887 12174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:10.661177 12174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:10.662335 12174 hybrid_clock.cc:648] HybridClock initialized: now 1781901610662314 us; error 34 us; skew 500 ppm
I20260619 20:40:10.663618 12174 webserver.cc:460] Webserver started at http://127.25.62.130:41701/ using document root <none> and password file <none>
I20260619 20:40:10.663885 12174 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:10.663959 12174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:10.664106 12174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:10.665493 12174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "a2659cc42ff34e60acc80bb2a6362613"
format_stamp: "Formatted at 2026-06-19 20:40:10 on dist-test-slave-0t40"
server_key: "da7becff1b1f3b30493b6e9b16b027f6"
server_key_iv: "30ac33c8e72e2c168370f20e7497d860"
server_key_version: "encryptionkey@0"
I20260619 20:40:10.665911 12174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "a2659cc42ff34e60acc80bb2a6362613"
format_stamp: "Formatted at 2026-06-19 20:40:10 on dist-test-slave-0t40"
server_key: "da7becff1b1f3b30493b6e9b16b027f6"
server_key_iv: "30ac33c8e72e2c168370f20e7497d860"
server_key_version: "encryptionkey@0"
I20260619 20:40:10.667523 12174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:40:10.668612 12188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:10.668799 12174 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:10.668898 12174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "a2659cc42ff34e60acc80bb2a6362613"
format_stamp: "Formatted at 2026-06-19 20:40:10 on dist-test-slave-0t40"
server_key: "da7becff1b1f3b30493b6e9b16b027f6"
server_key_iv: "30ac33c8e72e2c168370f20e7497d860"
server_key_version: "encryptionkey@0"
I20260619 20:40:10.668970 12174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:10.676800 12174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:10.677106 12174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:10.677248 12174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:10.677464 12174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:40:10.677763 12174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:40:10.677798 12174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:10.677819 12174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:40:10.677863 12174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:10.683262 12174 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:37349
I20260619 20:40:10.683300 12301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:37349 every 8 connection(s)
I20260619 20:40:10.683611 12174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:40:10.686033 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 12174
I20260619 20:40:10.686146 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:40:10.686338 25850 external_mini_cluster.cc:1428] Setting key f051c6d53135111a631144b13c9a0ddc
I20260619 20:40:10.687234 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:36339
--builtin_ntp_servers=127.25.62.148:43539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 20:40:10.689350 12023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.684357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:57791 (local address 127.25.62.190:36339)
0619 20:40:10.684435 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:10.684438 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:10.684613 (+ 175us) server_negotiation.cc:407] Connection header received
0619 20:40:10.685106 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:10.685107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:10.685119 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:10.685183 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:10.685801 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.686402 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.687808 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.687998 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.688054 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.688966 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:10.688967 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:10.688968 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:10.688973 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:10.688991 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:10.688995 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:10.689082 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:10.689205 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 20:40:10.689237 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 20:40:10.689455 12304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.684305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:36339 (local address 127.25.62.130:57791)
0619 20:40:10.684504 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:10.684535 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:10.684978 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:10.685255 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:10.685259 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:10.685266 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:10.685731 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.685735 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.686483 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.686484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.687728 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.687731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.688078 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.688079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.688234 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.688833 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:40:10.688834 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:10.688860 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:40:10.688880 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:10.689116 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:10.689121 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:10.689124 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:40:10.689189 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 20:40:10.689265 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":103,"threads_started":1}
I20260619 20:40:10.690795 12302 heartbeater.cc:347] Connected to a master server at 127.25.62.190:36339
I20260619 20:40:10.690884 12302 heartbeater.cc:464] Registering TS with master...
I20260619 20:40:10.691071 12302 heartbeater.cc:511] Master 127.25.62.190:36339 requested a full tablet report, sending...
I20260619 20:40:10.691461 11983 ts_manager.cc:194] Registered new tserver with Master: a2659cc42ff34e60acc80bb2a6362613 (127.25.62.130:37349)
I20260619 20:40:10.692008 11983 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.62.130:57791
W20260619 20:40:10.767890 12305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:10.768204 12305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:10.768241 12305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:10.769842 12305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:10.769882 12305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:10.769908 12305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:40:10.771251 12305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:43539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:36339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:10.771549 12305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:10.771883 12305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:10.776867 12313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:10.776883 12310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:10.776883 12311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:10.777248 12305 server_base.cc:1034] running on GCE node
I20260619 20:40:10.777483 12305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:10.777716 12305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:10.778873 12305 hybrid_clock.cc:648] HybridClock initialized: now 1781901610778860 us; error 27 us; skew 500 ppm
I20260619 20:40:10.784945 12305 webserver.cc:460] Webserver started at http://127.25.62.131:44781/ using document root <none> and password file <none>
I20260619 20:40:10.785238 12305 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:10.785312 12305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:10.785466 12305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:10.786561 12305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "3ef558b3e828456cad8fd104cfbb03b8"
format_stamp: "Formatted at 2026-06-19 20:40:10 on dist-test-slave-0t40"
server_key: "cdc4f2287f998ce706f9e3c537d88ef7"
server_key_iv: "d1c8197043f998dda50497fe3ec5b215"
server_key_version: "encryptionkey@0"
I20260619 20:40:10.786942 12305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "3ef558b3e828456cad8fd104cfbb03b8"
format_stamp: "Formatted at 2026-06-19 20:40:10 on dist-test-slave-0t40"
server_key: "cdc4f2287f998ce706f9e3c537d88ef7"
server_key_iv: "d1c8197043f998dda50497fe3ec5b215"
server_key_version: "encryptionkey@0"
I20260619 20:40:10.788278 12305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:40:10.789029 12319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:10.789273 12305 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:10.789368 12305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "3ef558b3e828456cad8fd104cfbb03b8"
format_stamp: "Formatted at 2026-06-19 20:40:10 on dist-test-slave-0t40"
server_key: "cdc4f2287f998ce706f9e3c537d88ef7"
server_key_iv: "d1c8197043f998dda50497fe3ec5b215"
server_key_version: "encryptionkey@0"
I20260619 20:40:10.789454 12305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:10.816730 12305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:10.817103 12305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:10.817267 12305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:10.817502 12305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:40:10.817838 12305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:40:10.817891 12305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:10.817935 12305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:40:10.817965 12305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:10.823877 12305 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:34795
I20260619 20:40:10.823920 12432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:34795 every 8 connection(s)
I20260619 20:40:10.824280 12305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:40:10.829208 12023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.825023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:58677 (local address 127.25.62.190:36339)
0619 20:40:10.825122 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:10.825124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:10.825357 (+ 233us) server_negotiation.cc:407] Connection header received
0619 20:40:10.825803 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:10.825804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:10.825814 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:10.825879 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:10.826508 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.827002 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.827926 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.828109 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.828164 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.828918 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:10.828919 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:10.828919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:10.828923 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:10.828937 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:10.828941 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:10.828989 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:10.829128 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 20:40:10.829151 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 20:40:10.829432 12435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.824994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:36339 (local address 127.25.62.131:58677)
0619 20:40:10.825233 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:10.825266 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:10.825725 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:10.825932 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:10.825935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:10.825943 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:10.826431 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.826435 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.827081 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.827082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.827845 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.827847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.828192 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.828193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.828309 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.828798 (+ 489us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:40:10.828798 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:10.828825 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:40:10.828844 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:10.829025 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:10.829030 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:10.829033 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:40:10.829124 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 20:40:10.829203 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":132,"threads_started":1}
I20260619 20:40:10.830003 12433 heartbeater.cc:347] Connected to a master server at 127.25.62.190:36339
I20260619 20:40:10.830077 12433 heartbeater.cc:464] Registering TS with master...
I20260619 20:40:10.830210 12433 heartbeater.cc:511] Master 127.25.62.190:36339 requested a full tablet report, sending...
I20260619 20:40:10.830523 11983 ts_manager.cc:194] Registered new tserver with Master: 3ef558b3e828456cad8fd104cfbb03b8 (127.25.62.131:34795)
I20260619 20:40:10.830895 11983 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.62.131:58677
I20260619 20:40:10.832684 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 12305
I20260619 20:40:10.832756 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:40:10.832948 25850 external_mini_cluster.cc:1428] Setting key e7eed80255b3a6cd2cd3c9ef1df2a4dd
I20260619 20:40:10.833904 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:40:10.838152 12023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.835254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54212 (local address 127.25.62.190:36339)
0619 20:40:10.835349 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:10.835351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:10.835416 (+ 65us) server_negotiation.cc:407] Connection header received
0619 20:40:10.835490 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:10.835490 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:10.835502 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:10.835575 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:10.835748 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.836195 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.836968 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.837170 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.837247 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.837873 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:10.837875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:10.837876 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:10.837880 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:10.837892 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:10.837896 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:10.837992 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:10.838058 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 20:40:10.838068 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 20:40:10.839594 11983 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:40:10.841095 11983 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:40:10.847136 12442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.843168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:34795 (local address 127.0.0.1:36534)
0619 20:40:10.843555 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:10.843563 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:10.843601 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:10.843821 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:10.843824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:10.843826 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:10.843886 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.843888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.844806 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.844807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.845961 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.845964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.846847 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.846848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.846918 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.846931 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 20:40:10.847012 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 20:40:10.847060 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":99,"threads_started":1}
I20260619 20:40:10.847146 12444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.843401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36534 (local address 127.25.62.131:34795)
0619 20:40:10.843676 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:10.843679 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:10.843686 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:40:10.843712 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:10.843713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:10.843724 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:10.843808 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:10.843968 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.844712 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.846056 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.846761 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.846837 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.846922 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:10.847027 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 20:40:10.847065 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":70,"threads_started":1}
I20260619 20:40:10.847383 12441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.843168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:33145 (local address 127.0.0.1:41074)
0619 20:40:10.843639 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:10.843647 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:10.843726 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:10.843989 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:10.843990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:10.843992 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:10.844027 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.844028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.845391 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.845393 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.846189 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.846192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.847159 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.847160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.847229 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.847234 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:40:10.847286 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 20:40:10.847333 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":3,"thread_start_us":95,"threads_started":1}
I20260619 20:40:10.847503 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.843156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:37349 (local address 127.0.0.1:38804)
0619 20:40:10.843326 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:10.843383 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:10.843449 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:10.844012 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:10.844013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:10.844015 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:10.844061 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.844062 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.845375 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.845376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.846432 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.846434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.847219 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.847220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.847313 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.847317 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:40:10.847346 (+ 29us) client_negotiation.cc:240] Negotiation successful
0619 20:40:10.847380 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":25,"thread_start_us":50,"threads_started":1}
I20260619 20:40:10.847640 12446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.843413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41074 (local address 127.25.62.129:33145)
0619 20:40:10.843595 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:10.843598 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:10.843718 (+ 120us) server_negotiation.cc:407] Connection header received
0619 20:40:10.843795 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:10.843798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:10.843814 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:10.843897 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:10.844138 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.845307 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.846284 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.846984 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.847046 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.847122 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:10.847504 (+ 382us) server_negotiation.cc:299] Negotiation successful
0619 20:40:10.847546 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":79,"threads_started":1}
I20260619 20:40:10.847673 12445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.843412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38804 (local address 127.25.62.130:37349)
0619 20:40:10.843852 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:10.843856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:10.843865 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:40:10.843900 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:10.843901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:10.843916 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:10.844136 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:10.844198 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.845286 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.846514 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.847125 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.847417 (+ 292us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.847516 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:10.847544 (+ 28us) server_negotiation.cc:299] Negotiation successful
0619 20:40:10.847594 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":82,"threads_started":1}
I20260619 20:40:10.848251 12105 tablet_service.cc:1467] Processing CreateTablet for tablet 6d909ddef2904aae8e5c264a113f03e9 (DEFAULT_TABLE table=test-table [id=af00119ceff347ad9d157bb548ead034]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:40:10.848250 12367 tablet_service.cc:1467] Processing CreateTablet for tablet 6d909ddef2904aae8e5c264a113f03e9 (DEFAULT_TABLE table=test-table [id=af00119ceff347ad9d157bb548ead034]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:40:10.848601 12367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d909ddef2904aae8e5c264a113f03e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:10.848601 12105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d909ddef2904aae8e5c264a113f03e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:10.849407 12236 tablet_service.cc:1467] Processing CreateTablet for tablet 6d909ddef2904aae8e5c264a113f03e9 (DEFAULT_TABLE table=test-table [id=af00119ceff347ad9d157bb548ead034]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:40:10.849725 12236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d909ddef2904aae8e5c264a113f03e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:10.851940 12448 tablet_bootstrap.cc:492] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8: Bootstrap starting.
I20260619 20:40:10.852662 12448 tablet_bootstrap.cc:654] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:10.852837 12449 tablet_bootstrap.cc:492] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613: Bootstrap starting.
I20260619 20:40:10.853158 12448 log.cc:824] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:10.853205 12447 tablet_bootstrap.cc:492] T 6d909ddef2904aae8e5c264a113f03e9 P 984cbec8aeff4c2c8024852e5835bacb: Bootstrap starting.
I20260619 20:40:10.853507 12449 tablet_bootstrap.cc:654] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:10.853799 12447 tablet_bootstrap.cc:654] T 6d909ddef2904aae8e5c264a113f03e9 P 984cbec8aeff4c2c8024852e5835bacb: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:10.853804 12449 log.cc:824] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:10.853960 12448 tablet_bootstrap.cc:492] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8: No bootstrap required, opened a new log
I20260619 20:40:10.854030 12448 ts_tablet_manager.cc:1397] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 20:40:10.854118 12447 log.cc:824] T 6d909ddef2904aae8e5c264a113f03e9 P 984cbec8aeff4c2c8024852e5835bacb: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:10.854564 12449 tablet_bootstrap.cc:492] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613: No bootstrap required, opened a new log
I20260619 20:40:10.854633 12449 ts_tablet_manager.cc:1397] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:40:10.854902 12447 tablet_bootstrap.cc:492] T 6d909ddef2904aae8e5c264a113f03e9 P 984cbec8aeff4c2c8024852e5835bacb: No bootstrap required, opened a new log
I20260619 20:40:10.854966 12447 ts_tablet_manager.cc:1397] T 6d909ddef2904aae8e5c264a113f03e9 P 984cbec8aeff4c2c8024852e5835bacb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:40:10.856460 12449 raft_consensus.cc:348] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984cbec8aeff4c2c8024852e5835bacb" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33145 } } peers { permanent_uuid: "a2659cc42ff34e60acc80bb2a6362613" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37349 } } peers { permanent_uuid: "3ef558b3e828456cad8fd104cfbb03b8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34795 } }
I20260619 20:40:10.856657 12449 raft_consensus.cc:374] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:10.856696 12449 raft_consensus.cc:729] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2659cc42ff34e60acc80bb2a6362613, State: Initialized, Role: FOLLOWER
I20260619 20:40:10.856807 12449 consensus_queue.cc:260] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984cbec8aeff4c2c8024852e5835bacb" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33145 } } peers { permanent_uuid: "a2659cc42ff34e60acc80bb2a6362613" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37349 } } peers { permanent_uuid: "3ef558b3e828456cad8fd104cfbb03b8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34795 } }
I20260619 20:40:10.856772 12448 raft_consensus.cc:348] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984cbec8aeff4c2c8024852e5835bacb" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33145 } } peers { permanent_uuid: "a2659cc42ff34e60acc80bb2a6362613" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37349 } } peers { permanent_uuid: "3ef558b3e828456cad8fd104cfbb03b8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34795 } }
I20260619 20:40:10.856928 12448 raft_consensus.cc:374] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:10.856962 12448 raft_consensus.cc:729] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ef558b3e828456cad8fd104cfbb03b8, State: Initialized, Role: FOLLOWER
I20260619 20:40:10.857076 12449 ts_tablet_manager.cc:1428] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 20:40:10.857110 12448 consensus_queue.cc:260] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984cbec8aeff4c2c8024852e5835bacb" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33145 } } peers { permanent_uuid: "a2659cc42ff34e60acc80bb2a6362613" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37349 } } peers { permanent_uuid: "3ef558b3e828456cad8fd104cfbb03b8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34795 } }
I20260619 20:40:10.857188 12302 heartbeater.cc:503] Master 127.25.62.190:36339 was elected leader, sending a full tablet report...
I20260619 20:40:10.857367 12433 heartbeater.cc:503] Master 127.25.62.190:36339 was elected leader, sending a full tablet report...
I20260619 20:40:10.857352 12448 ts_tablet_manager.cc:1428] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 20:40:10.857618 12447 raft_consensus.cc:348] T 6d909ddef2904aae8e5c264a113f03e9 P 984cbec8aeff4c2c8024852e5835bacb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984cbec8aeff4c2c8024852e5835bacb" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33145 } } peers { permanent_uuid: "a2659cc42ff34e60acc80bb2a6362613" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37349 } } peers { permanent_uuid: "3ef558b3e828456cad8fd104cfbb03b8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34795 } }
I20260619 20:40:10.857769 12447 raft_consensus.cc:374] T 6d909ddef2904aae8e5c264a113f03e9 P 984cbec8aeff4c2c8024852e5835bacb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:10.857812 12447 raft_consensus.cc:729] T 6d909ddef2904aae8e5c264a113f03e9 P 984cbec8aeff4c2c8024852e5835bacb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984cbec8aeff4c2c8024852e5835bacb, State: Initialized, Role: FOLLOWER
I20260619 20:40:10.857908 12447 consensus_queue.cc:260] T 6d909ddef2904aae8e5c264a113f03e9 P 984cbec8aeff4c2c8024852e5835bacb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984cbec8aeff4c2c8024852e5835bacb" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33145 } } peers { permanent_uuid: "a2659cc42ff34e60acc80bb2a6362613" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37349 } } peers { permanent_uuid: "3ef558b3e828456cad8fd104cfbb03b8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34795 } }
I20260619 20:40:10.858163 12447 ts_tablet_manager.cc:1428] T 6d909ddef2904aae8e5c264a113f03e9 P 984cbec8aeff4c2c8024852e5835bacb: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 20:40:10.858248 12171 heartbeater.cc:503] Master 127.25.62.190:36339 was elected leader, sending a full tablet report...
I20260619 20:40:10.910578 12454 raft_consensus.cc:482] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:40:10.910744 12454 raft_consensus.cc:504] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984cbec8aeff4c2c8024852e5835bacb" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33145 } } peers { permanent_uuid: "a2659cc42ff34e60acc80bb2a6362613" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37349 } } peers { permanent_uuid: "3ef558b3e828456cad8fd104cfbb03b8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34795 } }
I20260619 20:40:10.911146 12454 leader_election.cc:288] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 984cbec8aeff4c2c8024852e5835bacb (127.25.62.129:33145), a2659cc42ff34e60acc80bb2a6362613 (127.25.62.130:37349)
I20260619 20:40:10.915086 12445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.911456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:56643 (local address 127.25.62.130:37349)
0619 20:40:10.911510 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:10.911512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:10.911758 (+ 246us) server_negotiation.cc:407] Connection header received
0619 20:40:10.911820 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:10.911821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:10.911831 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:10.911887 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:10.911991 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.912665 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.914109 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.914686 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.914771 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.914780 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:10.914973 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 20:40:10.915015 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 20:40:10.915086 12446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.911419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:42361 (local address 127.25.62.129:33145)
0619 20:40:10.911494 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:10.911497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:10.911508 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:40:10.911536 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:10.911537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:10.911550 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:10.911613 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:10.911826 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.912571 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.914110 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.914686 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.914774 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.914780 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:10.914973 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 20:40:10.915015 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 20:40:10.915089 12435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.911328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:33145 (local address 127.25.62.131:42361)
0619 20:40:10.911391 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:10.911406 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:10.911461 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:10.911645 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:10.911646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:10.911648 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:10.911745 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.911747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.912656 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.912657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.913969 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.913972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.914799 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.914800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.914881 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.914885 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:40:10.914948 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 20:40:10.914990 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":51}
I20260619 20:40:10.915107 12456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.911330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:37349 (local address 127.25.62.131:56643)
0619 20:40:10.911679 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:10.911690 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:10.911748 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:10.911899 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:10.911900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:10.911901 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:10.911948 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.911949 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.912743 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.912743 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.913969 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:10.913972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.914799 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.914800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:10.914886 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.914888 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:40:10.914956 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 20:40:10.914989 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":161,"thread_start_us":115,"threads_started":1}
I20260619 20:40:10.915530 12125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d909ddef2904aae8e5c264a113f03e9" candidate_uuid: "3ef558b3e828456cad8fd104cfbb03b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984cbec8aeff4c2c8024852e5835bacb" is_pre_election: true
I20260619 20:40:10.915616 12256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d909ddef2904aae8e5c264a113f03e9" candidate_uuid: "3ef558b3e828456cad8fd104cfbb03b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2659cc42ff34e60acc80bb2a6362613" is_pre_election: true
I20260619 20:40:10.915716 12125 raft_consensus.cc:2454] T 6d909ddef2904aae8e5c264a113f03e9 P 984cbec8aeff4c2c8024852e5835bacb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef558b3e828456cad8fd104cfbb03b8 in term 0.
I20260619 20:40:10.915750 12256 raft_consensus.cc:2454] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ef558b3e828456cad8fd104cfbb03b8 in term 0.
I20260619 20:40:10.915946 12322 leader_election.cc:302] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef558b3e828456cad8fd104cfbb03b8, 984cbec8aeff4c2c8024852e5835bacb; no voters:
I20260619 20:40:10.916113 12454 raft_consensus.cc:2781] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:40:10.916172 12454 raft_consensus.cc:482] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:40:10.916200 12454 raft_consensus.cc:3037] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:10.916868 12454 raft_consensus.cc:504] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984cbec8aeff4c2c8024852e5835bacb" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33145 } } peers { permanent_uuid: "a2659cc42ff34e60acc80bb2a6362613" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37349 } } peers { permanent_uuid: "3ef558b3e828456cad8fd104cfbb03b8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34795 } }
I20260619 20:40:10.917004 12454 leader_election.cc:288] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [CANDIDATE]: Term 1 election: Requested vote from peers 984cbec8aeff4c2c8024852e5835bacb (127.25.62.129:33145), a2659cc42ff34e60acc80bb2a6362613 (127.25.62.130:37349)
I20260619 20:40:10.917168 12125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d909ddef2904aae8e5c264a113f03e9" candidate_uuid: "3ef558b3e828456cad8fd104cfbb03b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984cbec8aeff4c2c8024852e5835bacb"
I20260619 20:40:10.917268 12256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d909ddef2904aae8e5c264a113f03e9" candidate_uuid: "3ef558b3e828456cad8fd104cfbb03b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2659cc42ff34e60acc80bb2a6362613"
I20260619 20:40:10.917259 12125 raft_consensus.cc:3037] T 6d909ddef2904aae8e5c264a113f03e9 P 984cbec8aeff4c2c8024852e5835bacb [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:10.917349 12256 raft_consensus.cc:3037] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:10.918236 12125 raft_consensus.cc:2454] T 6d909ddef2904aae8e5c264a113f03e9 P 984cbec8aeff4c2c8024852e5835bacb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef558b3e828456cad8fd104cfbb03b8 in term 1.
I20260619 20:40:10.918260 12256 raft_consensus.cc:2454] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ef558b3e828456cad8fd104cfbb03b8 in term 1.
I20260619 20:40:10.918411 12322 leader_election.cc:302] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ef558b3e828456cad8fd104cfbb03b8, 984cbec8aeff4c2c8024852e5835bacb; no voters:
I20260619 20:40:10.918574 12454 raft_consensus.cc:2781] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:10.918761 12454 raft_consensus.cc:686] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [term 1 LEADER]: Becoming Leader. State: Replica: 3ef558b3e828456cad8fd104cfbb03b8, State: Running, Role: LEADER
I20260619 20:40:10.918872 12454 consensus_queue.cc:237] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984cbec8aeff4c2c8024852e5835bacb" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33145 } } peers { permanent_uuid: "a2659cc42ff34e60acc80bb2a6362613" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37349 } } peers { permanent_uuid: "3ef558b3e828456cad8fd104cfbb03b8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34795 } }
I20260619 20:40:10.919678 11983 catalog_manager.cc:5495] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ef558b3e828456cad8fd104cfbb03b8 (127.25.62.131). New cstate: current_term: 1 leader_uuid: "3ef558b3e828456cad8fd104cfbb03b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "984cbec8aeff4c2c8024852e5835bacb" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2659cc42ff34e60acc80bb2a6362613" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ef558b3e828456cad8fd104cfbb03b8" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 34795 } health_report { overall_health: HEALTHY } } }
W20260619 20:40:10.934134 12303 tablet.cc:2369] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:40:10.938931 12444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.936690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36540 (local address 127.25.62.131:34795)
0619 20:40:10.936761 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:10.936763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:10.936770 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:40:10.936805 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:10.936806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:10.936824 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:10.936883 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:40:10.937034 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.937545 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.938246 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:10.938426 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:10.938476 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:10.938609 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:40:10.938689 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:40:10.938757 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:10.938800 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 20:40:10.938807 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 20:40:10.940660 12256 raft_consensus.cc:1261] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613 [term 1 FOLLOWER]: Refusing update from remote peer 3ef558b3e828456cad8fd104cfbb03b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:40:10.940675 12125 raft_consensus.cc:1261] T 6d909ddef2904aae8e5c264a113f03e9 P 984cbec8aeff4c2c8024852e5835bacb [term 1 FOLLOWER]: Refusing update from remote peer 3ef558b3e828456cad8fd104cfbb03b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:40:10.941066 12457 consensus_queue.cc:1035] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984cbec8aeff4c2c8024852e5835bacb" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 33145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:40:10.941203 12454 consensus_queue.cc:1035] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2659cc42ff34e60acc80bb2a6362613" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 37349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:40:10.943418 12460 mvcc.cc:205] Tried to move back new op lower bound from 7298668998410264576 to 7298668998324428800. Current Snapshot: MvccSnapshot[applied={T|T < 7298668998410264576}]
I20260619 20:40:10.944144 12462 mvcc.cc:205] Tried to move back new op lower bound from 7298668998410264576 to 7298668998324428800. Current Snapshot: MvccSnapshot[applied={T|T < 7298668998410264576}]
I20260619 20:40:10.944430 12461 mvcc.cc:205] Tried to move back new op lower bound from 7298668998410264576 to 7298668998324428800. Current Snapshot: MvccSnapshot[applied={T|T < 7298668998410264576 or (T in {7298668998410264576})}]
I20260619 20:40:10.946489 11983 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:40:10.946576 11983 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:40:10.947351 11983 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87001c14e0e148de871079373b9074e3: Sending DeleteTablet for 3 replicas of tablet 6d909ddef2904aae8e5c264a113f03e9
I20260619 20:40:10.947666 12105 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d909ddef2904aae8e5c264a113f03e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:40:10 UTC) from {username='slave'} at 127.0.0.1:41074
I20260619 20:40:10.947656 12236 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d909ddef2904aae8e5c264a113f03e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:40:10 UTC) from {username='slave'} at 127.0.0.1:38804
I20260619 20:40:10.947903 12469 tablet_replica.cc:330] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613: stopping tablet replica
I20260619 20:40:10.947904 12470 tablet_replica.cc:330] T 6d909ddef2904aae8e5c264a113f03e9 P 984cbec8aeff4c2c8024852e5835bacb: stopping tablet replica
I20260619 20:40:10.948009 12469 raft_consensus.cc:2229] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:40:10.948077 12367 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d909ddef2904aae8e5c264a113f03e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:40:10 UTC) from {username='slave'} at 127.0.0.1:36534
I20260619 20:40:10.948128 12469 raft_consensus.cc:2258] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:40:10.948105 12470 raft_consensus.cc:2229] T 6d909ddef2904aae8e5c264a113f03e9 P 984cbec8aeff4c2c8024852e5835bacb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:40:10.948305 12470 raft_consensus.cc:2258] T 6d909ddef2904aae8e5c264a113f03e9 P 984cbec8aeff4c2c8024852e5835bacb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:40:10.948455 12471 tablet_replica.cc:330] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8: stopping tablet replica
I20260619 20:40:10.948544 12471 raft_consensus.cc:2229] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:40:10.948539 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 12043
I20260619 20:40:10.948575 12469 ts_tablet_manager.cc:1905] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:40:10.948644 12471 raft_consensus.cc:2258] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:40:10.949072 12471 ts_tablet_manager.cc:1905] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:40:10.950470 12469 ts_tablet_manager.cc:1918] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:40:10.950531 12469 log.cc:1196] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/6d909ddef2904aae8e5c264a113f03e9
I20260619 20:40:10.950780 12469 ts_tablet_manager.cc:1939] T 6d909ddef2904aae8e5c264a113f03e9 P a2659cc42ff34e60acc80bb2a6362613: Deleting consensus metadata
I20260619 20:40:10.951001 12471 ts_tablet_manager.cc:1918] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:40:10.951057 12471 log.cc:1196] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/6d909ddef2904aae8e5c264a113f03e9
I20260619 20:40:10.951164 11970 catalog_manager.cc:4841] TS a2659cc42ff34e60acc80bb2a6362613 (127.25.62.130:37349): tablet 6d909ddef2904aae8e5c264a113f03e9 (table test-table [id=af00119ceff347ad9d157bb548ead034]) successfully deleted
I20260619 20:40:10.951321 12471 ts_tablet_manager.cc:1939] T 6d909ddef2904aae8e5c264a113f03e9 P 3ef558b3e828456cad8fd104cfbb03b8: Deleting consensus metadata
I20260619 20:40:10.951685 11967 catalog_manager.cc:4841] TS 3ef558b3e828456cad8fd104cfbb03b8 (127.25.62.131:34795): tablet 6d909ddef2904aae8e5c264a113f03e9 (table test-table [id=af00119ceff347ad9d157bb548ead034]) successfully deleted
W20260619 20:40:10.956584 11969 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.62.129:33145 (error 108)
I20260619 20:40:10.956799 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 12174
I20260619 20:40:10.957026 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:10.956915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:33145 (local address 127.0.0.1:41084)
0619 20:40:10.956958 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:10.956989 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.129:33145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 20:40:10.957536 11969 catalog_manager.cc:4584] TS 984cbec8aeff4c2c8024852e5835bacb (127.25.62.129:33145): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6d909ddef2904aae8e5c264a113f03e9: Network error: Client connection negotiation failed: client connection to 127.25.62.129:33145: connect: Connection refused (error 111)
I20260619 20:40:10.961634 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 12305
I20260619 20:40:10.965931 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 11952
2026-06-19T20:40:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (698 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:11 dist-test-slave-0t40 krb5kdc[12475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:11 dist-test-slave-0t40 krb5kdc[12475](info): set up 2 sockets
Jun 19 20:40:11 dist-test-slave-0t40 krb5kdc[12475](info): commencing operation
krb5kdc: starting...
W20260619 20:40:12.009263 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:12 dist-test-slave-0t40 krb5kdc[12475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:40:12.126080 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:44937
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:44937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:12.205742 12488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:12.205940 12488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:12.205962 12488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:12.207008 12488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:12.207039 12488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:12.207054 12488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:12.207067 12488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:12.207079 12488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:12.208420 12488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:44937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:44937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:12.208595 12488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:12.208748 12488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:12.213825 12494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:12.213874 12493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:12.213863 12488 server_base.cc:1034] running on GCE node
W20260619 20:40:12.213831 12496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:12.214313 12488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:12.214656 12488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:12.215889 12488 hybrid_clock.cc:648] HybridClock initialized: now 1781901612215862 us; error 40 us; skew 500 ppm
Jun 19 20:40:12 dist-test-slave-0t40 krb5kdc[12475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901612, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:12.218645 12488 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:40:12.219076 12488 webserver.cc:460] Webserver started at http://127.25.62.190:33271/ using document root <none> and password file <none>
I20260619 20:40:12.219314 12488 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:12.219383 12488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:12.219529 12488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:12.220628 12488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "894f7b68c9da449a80fe3fe298fabce4"
format_stamp: "Formatted at 2026-06-19 20:40:12 on dist-test-slave-0t40"
server_key: "18dbb20175755df2c37ed44f7ddd88ac"
server_key_iv: "6edcc50db60722ea26f3e14ba0051cd6"
server_key_version: "encryptionkey@0"
I20260619 20:40:12.220986 12488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "894f7b68c9da449a80fe3fe298fabce4"
format_stamp: "Formatted at 2026-06-19 20:40:12 on dist-test-slave-0t40"
server_key: "18dbb20175755df2c37ed44f7ddd88ac"
server_key_iv: "6edcc50db60722ea26f3e14ba0051cd6"
server_key_version: "encryptionkey@0"
I20260619 20:40:12.222286 12488 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:12.223062 12503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:12.223230 12488 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:12.223330 12488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "894f7b68c9da449a80fe3fe298fabce4"
format_stamp: "Formatted at 2026-06-19 20:40:12 on dist-test-slave-0t40"
server_key: "18dbb20175755df2c37ed44f7ddd88ac"
server_key_iv: "6edcc50db60722ea26f3e14ba0051cd6"
server_key_version: "encryptionkey@0"
I20260619 20:40:12.223403 12488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:12.245004 12488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:12.247148 12488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:12.247313 12488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:12.251310 12488 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:44937
I20260619 20:40:12.251369 12555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:44937 every 8 connection(s)
I20260619 20:40:12.251727 12488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:12.252607 12556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:12.255115 12556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4: Bootstrap starting.
I20260619 20:40:12.255831 12556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:12.256102 12556 log.cc:824] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:12.256803 12556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4: No bootstrap required, opened a new log
I20260619 20:40:12.258380 12556 raft_consensus.cc:348] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "894f7b68c9da449a80fe3fe298fabce4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44937 } }
I20260619 20:40:12.258536 12556 raft_consensus.cc:374] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:12.258582 12556 raft_consensus.cc:729] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 894f7b68c9da449a80fe3fe298fabce4, State: Initialized, Role: FOLLOWER
I20260619 20:40:12.258764 12556 consensus_queue.cc:260] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "894f7b68c9da449a80fe3fe298fabce4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44937 } }
I20260619 20:40:12.258863 12556 raft_consensus.cc:388] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:12.258905 12556 raft_consensus.cc:482] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:12.258955 12556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:12.259476 12556 raft_consensus.cc:504] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "894f7b68c9da449a80fe3fe298fabce4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44937 } }
I20260619 20:40:12.259590 12556 leader_election.cc:302] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 894f7b68c9da449a80fe3fe298fabce4; no voters:
I20260619 20:40:12.259785 12556 leader_election.cc:288] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:12.259826 12559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:12.260036 12556 sys_catalog.cc:564] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:12.260116 12559 raft_consensus.cc:686] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4 [term 1 LEADER]: Becoming Leader. State: Replica: 894f7b68c9da449a80fe3fe298fabce4, State: Running, Role: LEADER
I20260619 20:40:12.260213 12559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "894f7b68c9da449a80fe3fe298fabce4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44937 } }
I20260619 20:40:12.260622 12560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "894f7b68c9da449a80fe3fe298fabce4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "894f7b68c9da449a80fe3fe298fabce4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44937 } } }
I20260619 20:40:12.260782 12560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:12.260852 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 12488
I20260619 20:40:12.260969 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:12.260928 12561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 894f7b68c9da449a80fe3fe298fabce4. Latest consensus state: current_term: 1 leader_uuid: "894f7b68c9da449a80fe3fe298fabce4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "894f7b68c9da449a80fe3fe298fabce4" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44937 } } }
I20260619 20:40:12.261015 12561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:12.261072 12568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:12.261201 25850 external_mini_cluster.cc:1428] Setting key 32f1982b5f5f77d8e954fe6557f7a286
I20260619 20:40:12.261580 12568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:12.263188 12568 catalog_manager.cc:1347] Generated new cluster ID: f5b7026ed83a428fbb5166764aef0d32
I20260619 20:40:12.263239 12568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:40:12 dist-test-slave-0t40 krb5kdc[12475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:40:12.273083 12577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.261898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41026 (local address 127.25.62.190:44937)
0619 20:40:12.262268 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:12.262289 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:12.262322 (+ 33us) server_negotiation.cc:407] Connection header received
0619 20:40:12.262434 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:12.262440 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:12.262495 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:12.262635 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:12.263432 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.264831 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.265866 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.266491 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.266596 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:12.269433 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:12.269437 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:12.269437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:12.269448 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:12.271352 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:12.271748 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:12.271751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:12.271752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:12.271869 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:12.272072 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:12.272072 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:12.272073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:12.272358 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:12.272497 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:12.272615 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 20:40:12.272841 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":58,"threads_started":1}
I20260619 20:40:12.302382 12568 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:12.303102 12568 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:12.308466 12568 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4: Generated new TSK 0
I20260619 20:40:12.308641 12568 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.25.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
Entry for principal kudu/127.25.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab.
I20260619 20:40:12.336357 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:44937
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:12.416939 12582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:12.417270 12582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:12.417308 12582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:12.418577 12582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:12.418619 12582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:12.418646 12582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:40:12.420193 12582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.129.keytab
--principal=kudu/127.25.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:44937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:12.420437 12582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:12.420688 12582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:12.426033 12590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:12.426054 12587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:12.426047 12588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:12.426290 12582 server_base.cc:1034] running on GCE node
I20260619 20:40:12.426607 12582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:12.426913 12582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:12.428090 12582 hybrid_clock.cc:648] HybridClock initialized: now 1781901612428064 us; error 36 us; skew 500 ppm
Jun 19 20:40:12 dist-test-slave-0t40 krb5kdc[12475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901612, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:12.430878 12582 init.cc:377] Logged in from keytab as kudu/127.25.62.129@KRBTEST.COM (short username kudu)
I20260619 20:40:12.431304 12582 webserver.cc:460] Webserver started at http://127.25.62.129:43461/ using document root <none> and password file <none>
I20260619 20:40:12.431507 12582 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:12.431552 12582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:12.431646 12582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:12.432674 12582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "09721578ee264c6ba2865a75376fa85a"
format_stamp: "Formatted at 2026-06-19 20:40:12 on dist-test-slave-0t40"
server_key: "13fff8ea7ba4906418550bab4f936bcf"
server_key_iv: "e178305d421d279f8819f667b6669fb4"
server_key_version: "encryptionkey@0"
I20260619 20:40:12.433015 12582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "09721578ee264c6ba2865a75376fa85a"
format_stamp: "Formatted at 2026-06-19 20:40:12 on dist-test-slave-0t40"
server_key: "13fff8ea7ba4906418550bab4f936bcf"
server_key_iv: "e178305d421d279f8819f667b6669fb4"
server_key_version: "encryptionkey@0"
I20260619 20:40:12.434448 12582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:12.435199 12597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:12.435397 12582 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:12.435493 12582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "09721578ee264c6ba2865a75376fa85a"
format_stamp: "Formatted at 2026-06-19 20:40:12 on dist-test-slave-0t40"
server_key: "13fff8ea7ba4906418550bab4f936bcf"
server_key_iv: "e178305d421d279f8819f667b6669fb4"
server_key_version: "encryptionkey@0"
I20260619 20:40:12.435576 12582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:12.450925 12582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:12.452957 12582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:12.453157 12582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:12.453405 12582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:40:12.453719 12582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:40:12.453774 12582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:12.453814 12582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:40:12.453851 12582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:12.459792 12582 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:34353
I20260619 20:40:12.459830 12710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:34353 every 8 connection(s)
I20260619 20:40:12.460194 12582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:40:12.461190 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 12582
I20260619 20:40:12.461326 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:40:12.461565 25850 external_mini_cluster.cc:1428] Setting key 39d5d2c0518eba4e327f218165b941e5
Jun 19 20:40:12 dist-test-slave-0t40 krb5kdc[12475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901612, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.129@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.62.130@KRBTEST.COM; defaulting to no policy
I20260619 20:40:12.471387 12577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.461952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:33045 (local address 127.25.62.190:44937)
0619 20:40:12.461996 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:12.461998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:12.462258 (+ 260us) server_negotiation.cc:407] Connection header received
0619 20:40:12.462818 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:12.462819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:12.462835 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:12.462910 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:12.464034 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.464523 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.465733 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.466111 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.466192 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:12.468759 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:12.468760 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:12.468761 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:12.468767 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:12.470148 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:12.470651 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:12.470652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:12.470652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:12.470701 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:12.470908 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:12.470909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:12.470909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:12.471002 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:12.471046 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:12.471249 (+ 203us) server_negotiation.cc:299] Negotiation successful
0619 20:40:12.471315 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:40:12.471647 12713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.461949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:44937 (local address 127.25.62.129:33045)
0619 20:40:12.462110 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:12.462147 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:12.462691 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:12.462960 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:12.462963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:12.463642 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:12.463932 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:12.463939 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.464623 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.464624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:12.465629 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:12.465632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.466198 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.466199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:12.466300 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:12.466859 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:12.466867 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:12.468667 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:12.470233 (+ 1566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:12.470240 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:12.470242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:12.470567 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:12.470760 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:12.470761 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:12.470761 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:12.470862 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:12.471048 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:12.471051 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:12.471160 (+ 109us) client_negotiation.cc:769] Sending connection context
0619 20:40:12.471237 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 20:40:12.471370 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260619 20:40:12.472222 12711 heartbeater.cc:347] Connected to a master server at 127.25.62.190:44937
I20260619 20:40:12.472308 12711 heartbeater.cc:464] Registering TS with master...
I20260619 20:40:12.472589 12711 heartbeater.cc:511] Master 127.25.62.190:44937 requested a full tablet report, sending...
I20260619 20:40:12.473176 12520 ts_manager.cc:194] Registered new tserver with Master: 09721578ee264c6ba2865a75376fa85a (127.25.62.129:34353)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.130@KRBTEST.COM" created.
I20260619 20:40:12.473819 12520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.129@KRBTEST.COM'} at 127.25.62.129:33045
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
Entry for principal kudu/127.25.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab.
I20260619 20:40:12.488112 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:44937
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:12.569813 12716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:12.570068 12716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:12.570108 12716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:12.571273 12716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:12.571352 12716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:12.571400 12716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:40:12.572837 12716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.130.keytab
--principal=kudu/127.25.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:44937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:12.573134 12716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:12.573395 12716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:12.578631 12721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:12.578646 12724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:12.578651 12722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:12.578976 12716 server_base.cc:1034] running on GCE node
I20260619 20:40:12.579187 12716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:12.579434 12716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:12.580591 12716 hybrid_clock.cc:648] HybridClock initialized: now 1781901612580582 us; error 40 us; skew 500 ppm
Jun 19 20:40:12 dist-test-slave-0t40 krb5kdc[12475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901612, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:12.583071 12716 init.cc:377] Logged in from keytab as kudu/127.25.62.130@KRBTEST.COM (short username kudu)
I20260619 20:40:12.583534 12716 webserver.cc:460] Webserver started at http://127.25.62.130:45147/ using document root <none> and password file <none>
I20260619 20:40:12.583771 12716 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:12.583835 12716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:12.583973 12716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:12.585001 12716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "d677185faaac4e13973f9a201b469fbb"
format_stamp: "Formatted at 2026-06-19 20:40:12 on dist-test-slave-0t40"
server_key: "8eff8433ad015b822b558de7babb22b6"
server_key_iv: "5cd5938be8d46ef4f2ebdef8d9891988"
server_key_version: "encryptionkey@0"
I20260619 20:40:12.585417 12716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "d677185faaac4e13973f9a201b469fbb"
format_stamp: "Formatted at 2026-06-19 20:40:12 on dist-test-slave-0t40"
server_key: "8eff8433ad015b822b558de7babb22b6"
server_key_iv: "5cd5938be8d46ef4f2ebdef8d9891988"
server_key_version: "encryptionkey@0"
I20260619 20:40:12.586707 12716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:12.587451 12731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:12.587651 12716 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:40:12.587734 12716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "d677185faaac4e13973f9a201b469fbb"
format_stamp: "Formatted at 2026-06-19 20:40:12 on dist-test-slave-0t40"
server_key: "8eff8433ad015b822b558de7babb22b6"
server_key_iv: "5cd5938be8d46ef4f2ebdef8d9891988"
server_key_version: "encryptionkey@0"
I20260619 20:40:12.587831 12716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:12.601223 12716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:12.603395 12716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:12.603546 12716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:12.603771 12716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:40:12.604071 12716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:40:12.604104 12716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:12.604147 12716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:40:12.604175 12716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:12.609614 12716 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:35619
I20260619 20:40:12.609678 12844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:35619 every 8 connection(s)
I20260619 20:40:12.609994 12716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:40:12.612476 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 12716
I20260619 20:40:12.612582 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:40:12.612766 25850 external_mini_cluster.cc:1428] Setting key a4d5ae19872b71a8017fa7cd9091089c
Jun 19 20:40:12 dist-test-slave-0t40 krb5kdc[12475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901612, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.130@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:40:12.620165 12577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.610716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:41713 (local address 127.25.62.190:44937)
0619 20:40:12.610783 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:12.610785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:12.611010 (+ 225us) server_negotiation.cc:407] Connection header received
0619 20:40:12.611451 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:12.611452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:12.611463 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:12.611528 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:12.612403 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.613192 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.614250 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.614595 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.614673 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:12.617311 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:12.617312 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:12.617313 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:12.617320 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:12.618750 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:12.619213 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:12.619214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:12.619215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:12.619272 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:12.619515 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:12.619516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:12.619516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:12.619647 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:12.619720 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:12.619986 (+ 266us) server_negotiation.cc:299] Negotiation successful
0619 20:40:12.620070 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 20:40:12.620364 12847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.610713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:44937 (local address 127.25.62.130:41713)
0619 20:40:12.610869 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:12.610908 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:12.611368 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:12.611580 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:12.611583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:12.612111 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:12.612306 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:12.612310 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.613293 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.613294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:12.614162 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:12.614165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.614730 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.614731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:12.614908 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:12.615409 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:12.615420 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:12.617215 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:12.618865 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:12.618872 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:12.618874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:12.619098 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:12.619341 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:12.619342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:12.619342 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:12.619437 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:12.619723 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:12.619730 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:12.619856 (+ 126us) client_negotiation.cc:769] Sending connection context
0619 20:40:12.619965 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 20:40:12.620130 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260619 20:40:12.620826 12845 heartbeater.cc:347] Connected to a master server at 127.25.62.190:44937
I20260619 20:40:12.620908 12845 heartbeater.cc:464] Registering TS with master...
I20260619 20:40:12.621147 12845 heartbeater.cc:511] Master 127.25.62.190:44937 requested a full tablet report, sending...
I20260619 20:40:12.621547 12520 ts_manager.cc:194] Registered new tserver with Master: d677185faaac4e13973f9a201b469fbb (127.25.62.130:35619)
WARNING: no policy specified for kudu/127.25.62.131@KRBTEST.COM; defaulting to no policy
I20260619 20:40:12.622043 12520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.130@KRBTEST.COM'} at 127.25.62.130:41713
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
Entry for principal kudu/127.25.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab.
I20260619 20:40:12.640508 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:44937
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:37055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:12.721824 12850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:12.722013 12850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:12.722036 12850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:12.723171 12850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:12.723212 12850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:12.723239 12850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:40:12.724659 12850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:37055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.131.keytab
--principal=kudu/127.25.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:44937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:12.724907 12850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:12.725186 12850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:12.730135 12858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:12.730124 12855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:12.730127 12856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:12.730392 12850 server_base.cc:1034] running on GCE node
I20260619 20:40:12.730590 12850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:12.730782 12850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:12.731916 12850 hybrid_clock.cc:648] HybridClock initialized: now 1781901612731893 us; error 43 us; skew 500 ppm
Jun 19 20:40:12 dist-test-slave-0t40 krb5kdc[12475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901612, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:12.734465 12850 init.cc:377] Logged in from keytab as kudu/127.25.62.131@KRBTEST.COM (short username kudu)
I20260619 20:40:12.734930 12850 webserver.cc:460] Webserver started at http://127.25.62.131:43329/ using document root <none> and password file <none>
I20260619 20:40:12.735118 12850 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:12.735160 12850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:12.735247 12850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:12.736227 12850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "6762781096b44bacb8e2beeacdf22fbf"
format_stamp: "Formatted at 2026-06-19 20:40:12 on dist-test-slave-0t40"
server_key: "14b6276b08262ff568289bf8974d61af"
server_key_iv: "e77a83f7c6cf278b04d8203a915cb0fb"
server_key_version: "encryptionkey@0"
I20260619 20:40:12.736521 12850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "6762781096b44bacb8e2beeacdf22fbf"
format_stamp: "Formatted at 2026-06-19 20:40:12 on dist-test-slave-0t40"
server_key: "14b6276b08262ff568289bf8974d61af"
server_key_iv: "e77a83f7c6cf278b04d8203a915cb0fb"
server_key_version: "encryptionkey@0"
I20260619 20:40:12.737910 12850 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 20:40:12.738761 12865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:12.739022 12850 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:12.739136 12850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "6762781096b44bacb8e2beeacdf22fbf"
format_stamp: "Formatted at 2026-06-19 20:40:12 on dist-test-slave-0t40"
server_key: "14b6276b08262ff568289bf8974d61af"
server_key_iv: "e77a83f7c6cf278b04d8203a915cb0fb"
server_key_version: "encryptionkey@0"
I20260619 20:40:12.739219 12850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:12.749179 12850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:12.751436 12850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:12.751575 12850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:12.751817 12850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:40:12.752113 12850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:40:12.752151 12850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:12.752172 12850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:40:12.752218 12850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:12.758042 12850 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:38275
I20260619 20:40:12.758141 12978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:38275 every 8 connection(s)
I20260619 20:40:12.758435 12850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
Jun 19 20:40:12 dist-test-slave-0t40 krb5kdc[12475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901612, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.131@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:40:12.765237 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 12850
I20260619 20:40:12.765353 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:40:12.765559 25850 external_mini_cluster.cc:1428] Setting key 3e9c0d41220c05df4202b1d2bd674b85
I20260619 20:40:12.767796 12577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.759176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:34491 (local address 127.25.62.190:44937)
0619 20:40:12.759243 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:12.759245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:12.759454 (+ 209us) server_negotiation.cc:407] Connection header received
0619 20:40:12.759894 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:12.759895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:12.759906 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:12.759978 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:12.761005 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.761736 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.762774 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.762977 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.763062 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:12.765145 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:12.765146 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:12.765147 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:12.765154 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:12.766738 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:12.767083 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:12.767084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:12.767084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:12.767133 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:12.767336 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:12.767337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:12.767337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:12.767431 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:12.767479 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:12.767680 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 20:40:12.767736 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 20:40:12.767995 12981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.759158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:44937 (local address 127.25.62.131:34491)
0619 20:40:12.759325 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:12.759366 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:12.759813 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:12.760029 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:12.760032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:12.760673 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:12.760893 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:12.760899 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.761859 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.761860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:12.762658 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:12.762661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.763071 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.763073 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:12.763197 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:12.763674 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:12.763681 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:12.765003 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:12.766809 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:12.766814 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:12.766816 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:12.767020 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:12.767187 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:12.767187 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:12.767188 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:12.767282 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:12.767483 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:12.767486 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:12.767591 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 20:40:12.767668 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 20:40:12.767780 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
I20260619 20:40:12.768456 12979 heartbeater.cc:347] Connected to a master server at 127.25.62.190:44937
I20260619 20:40:12.768558 12979 heartbeater.cc:464] Registering TS with master...
I20260619 20:40:12.768728 12979 heartbeater.cc:511] Master 127.25.62.190:44937 requested a full tablet report, sending...
I20260619 20:40:12.769196 12520 ts_manager.cc:194] Registered new tserver with Master: 6762781096b44bacb8e2beeacdf22fbf (127.25.62.131:38275)
I20260619 20:40:12.769577 12520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.25.62.131@KRBTEST.COM'} at 127.25.62.131:34491
I20260619 20:40:12.777330 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:40:12.784158 12577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.778633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52324 (local address 127.25.62.190:44937)
0619 20:40:12.778702 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:12.778720 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:12.778794 (+ 74us) server_negotiation.cc:407] Connection header received
0619 20:40:12.778830 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:12.778831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:12.778840 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:12.778902 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:12.779426 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.779888 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.780571 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.780752 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.780800 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:12.781997 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:12.781998 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:12.781998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:12.782004 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:12.783195 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:12.783517 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:12.783518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:12.783518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:12.783564 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:12.783737 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:12.783738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:12.783738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:12.783863 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:12.783934 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:12.784044 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 20:40:12.784100 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:40:12.785763 12520 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:40:12.787340 12520 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:40:12.793617 12988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.789581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:35619 (local address 127.0.0.1:41396)
0619 20:40:12.789948 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:12.789957 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:12.790001 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:12.790329 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:12.790330 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:12.790332 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:12.790412 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:12.790414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.791384 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.791385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:12.792466 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:12.792469 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.793324 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.793325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:12.793400 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:12.793414 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 20:40:12.793493 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 20:40:12.793544 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":32,"thread_start_us":43,"threads_started":1}
I20260619 20:40:12.793665 12992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.789740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41396 (local address 127.25.62.130:35619)
0619 20:40:12.789929 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:12.789933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:12.790062 (+ 129us) server_negotiation.cc:407] Connection header received
0619 20:40:12.790119 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:12.790121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:12.790139 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:12.790247 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:12.790499 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.791263 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.792568 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.793233 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.793316 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:12.793371 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:12.793520 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 20:40:12.793564 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":78,"threads_started":1}
I20260619 20:40:12.793978 12989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.789482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40256 (local address 127.25.62.131:38275)
0619 20:40:12.789687 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:12.789692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:12.789836 (+ 144us) server_negotiation.cc:407] Connection header received
0619 20:40:12.789929 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:12.789931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:12.789948 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:12.790016 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:12.790211 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.791312 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.792846 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.793441 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.793501 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:12.793550 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:12.793862 (+ 312us) server_negotiation.cc:299] Negotiation successful
0619 20:40:12.793897 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":103,"threads_started":1}
I20260619 20:40:12.794037 12987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.789390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:34353 (local address 127.0.0.1:38784)
0619 20:40:12.789681 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:12.789720 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:12.789795 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:12.790014 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:12.790017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:12.790019 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:12.790088 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:12.790090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.791416 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.791416 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:12.792754 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:12.792755 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.793742 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.793743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:12.793824 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:12.793835 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 20:40:12.793924 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 20:40:12.793966 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":85,"thread_start_us":54,"threads_started":1}
I20260619 20:40:12.794059 12991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.789570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38784 (local address 127.25.62.129:34353)
0619 20:40:12.789765 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:12.789770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:12.789781 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:40:12.789906 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:12.789908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:12.789925 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:12.790002 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:12.790172 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.791293 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.792836 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.793563 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.793638 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:12.793689 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:12.793943 (+ 254us) server_negotiation.cc:299] Negotiation successful
0619 20:40:12.793982 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":87,"threads_started":1}
I20260619 20:40:12.793810 12990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.789390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:38275 (local address 127.0.0.1:40256)
0619 20:40:12.789761 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:12.789773 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:12.789846 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:12.790084 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:12.790085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:12.790087 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:12.790131 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:12.790133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.791415 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.791416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:12.792749 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:12.792752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.793581 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.793582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:12.793640 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:12.793644 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:40:12.793694 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 20:40:12.793738 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":121,"thread_start_us":56,"threads_started":1}
I20260619 20:40:12.794737 12779 tablet_service.cc:1467] Processing CreateTablet for tablet 8e06e49c7d6347cf9d4414472d0a0ee9 (DEFAULT_TABLE table=test-table [id=af2b04e00760425c9bfaeb2158d1307b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:40:12.794885 12913 tablet_service.cc:1467] Processing CreateTablet for tablet 8e06e49c7d6347cf9d4414472d0a0ee9 (DEFAULT_TABLE table=test-table [id=af2b04e00760425c9bfaeb2158d1307b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:40:12.795112 12779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e06e49c7d6347cf9d4414472d0a0ee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:12.795195 12913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e06e49c7d6347cf9d4414472d0a0ee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:12.795130 12645 tablet_service.cc:1467] Processing CreateTablet for tablet 8e06e49c7d6347cf9d4414472d0a0ee9 (DEFAULT_TABLE table=test-table [id=af2b04e00760425c9bfaeb2158d1307b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:40:12.795414 12645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e06e49c7d6347cf9d4414472d0a0ee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:12.797725 12993 tablet_bootstrap.cc:492] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb: Bootstrap starting.
I20260619 20:40:12.798421 12995 tablet_bootstrap.cc:492] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf: Bootstrap starting.
I20260619 20:40:12.798460 12993 tablet_bootstrap.cc:654] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:12.798462 12994 tablet_bootstrap.cc:492] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 09721578ee264c6ba2865a75376fa85a: Bootstrap starting.
I20260619 20:40:12.798962 12993 log.cc:824] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:12.799116 12995 tablet_bootstrap.cc:654] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:12.799116 12994 tablet_bootstrap.cc:654] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 09721578ee264c6ba2865a75376fa85a: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:12.799724 12993 tablet_bootstrap.cc:492] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb: No bootstrap required, opened a new log
I20260619 20:40:12.799707 12994 log.cc:824] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 09721578ee264c6ba2865a75376fa85a: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:12.799731 12995 log.cc:824] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:12.799813 12993 ts_tablet_manager.cc:1397] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:40:12.800419 12995 tablet_bootstrap.cc:492] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf: No bootstrap required, opened a new log
I20260619 20:40:12.800490 12995 ts_tablet_manager.cc:1397] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:40:12.800686 12994 tablet_bootstrap.cc:492] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 09721578ee264c6ba2865a75376fa85a: No bootstrap required, opened a new log
I20260619 20:40:12.800791 12994 ts_tablet_manager.cc:1397] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 09721578ee264c6ba2865a75376fa85a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:40:12.801751 12993 raft_consensus.cc:348] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09721578ee264c6ba2865a75376fa85a" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34353 } } peers { permanent_uuid: "6762781096b44bacb8e2beeacdf22fbf" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 38275 } } peers { permanent_uuid: "d677185faaac4e13973f9a201b469fbb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35619 } }
I20260619 20:40:12.801919 12993 raft_consensus.cc:374] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:12.801987 12993 raft_consensus.cc:729] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d677185faaac4e13973f9a201b469fbb, State: Initialized, Role: FOLLOWER
I20260619 20:40:12.802117 12993 consensus_queue.cc:260] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09721578ee264c6ba2865a75376fa85a" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34353 } } peers { permanent_uuid: "6762781096b44bacb8e2beeacdf22fbf" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 38275 } } peers { permanent_uuid: "d677185faaac4e13973f9a201b469fbb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35619 } }
I20260619 20:40:12.802416 12993 ts_tablet_manager.cc:1428] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 20:40:12.802565 12845 heartbeater.cc:503] Master 127.25.62.190:44937 was elected leader, sending a full tablet report...
I20260619 20:40:12.802460 12995 raft_consensus.cc:348] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09721578ee264c6ba2865a75376fa85a" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34353 } } peers { permanent_uuid: "6762781096b44bacb8e2beeacdf22fbf" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 38275 } } peers { permanent_uuid: "d677185faaac4e13973f9a201b469fbb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35619 } }
I20260619 20:40:12.802611 12995 raft_consensus.cc:374] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:12.802642 12995 raft_consensus.cc:729] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6762781096b44bacb8e2beeacdf22fbf, State: Initialized, Role: FOLLOWER
I20260619 20:40:12.802750 12995 consensus_queue.cc:260] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09721578ee264c6ba2865a75376fa85a" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34353 } } peers { permanent_uuid: "6762781096b44bacb8e2beeacdf22fbf" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 38275 } } peers { permanent_uuid: "d677185faaac4e13973f9a201b469fbb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35619 } }
I20260619 20:40:12.802706 12994 raft_consensus.cc:348] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 09721578ee264c6ba2865a75376fa85a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09721578ee264c6ba2865a75376fa85a" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34353 } } peers { permanent_uuid: "6762781096b44bacb8e2beeacdf22fbf" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 38275 } } peers { permanent_uuid: "d677185faaac4e13973f9a201b469fbb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35619 } }
I20260619 20:40:12.802910 12994 raft_consensus.cc:374] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 09721578ee264c6ba2865a75376fa85a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:12.803004 12995 ts_tablet_manager.cc:1428] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 20:40:12.803002 12994 raft_consensus.cc:729] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 09721578ee264c6ba2865a75376fa85a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09721578ee264c6ba2865a75376fa85a, State: Initialized, Role: FOLLOWER
I20260619 20:40:12.803030 12979 heartbeater.cc:503] Master 127.25.62.190:44937 was elected leader, sending a full tablet report...
I20260619 20:40:12.803153 12994 consensus_queue.cc:260] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 09721578ee264c6ba2865a75376fa85a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09721578ee264c6ba2865a75376fa85a" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34353 } } peers { permanent_uuid: "6762781096b44bacb8e2beeacdf22fbf" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 38275 } } peers { permanent_uuid: "d677185faaac4e13973f9a201b469fbb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35619 } }
I20260619 20:40:12.803400 12994 ts_tablet_manager.cc:1428] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 09721578ee264c6ba2865a75376fa85a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 20:40:12.803503 12711 heartbeater.cc:503] Master 127.25.62.190:44937 was elected leader, sending a full tablet report...
W20260619 20:40:12.860641 12846 tablet.cc:2369] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:40:12.904368 13000 raft_consensus.cc:482] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:40:12.904541 13000 raft_consensus.cc:504] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09721578ee264c6ba2865a75376fa85a" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34353 } } peers { permanent_uuid: "6762781096b44bacb8e2beeacdf22fbf" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 38275 } } peers { permanent_uuid: "d677185faaac4e13973f9a201b469fbb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35619 } }
I20260619 20:40:12.904966 13000 leader_election.cc:288] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09721578ee264c6ba2865a75376fa85a (127.25.62.129:34353), d677185faaac4e13973f9a201b469fbb (127.25.62.130:35619)
I20260619 20:40:12.908164 12981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.905214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:34353 (local address 127.25.62.131:54313)
0619 20:40:12.905286 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:12.905300 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:12.905387 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:12.905561 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:12.905563 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:12.905565 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:12.905677 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:12.905680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.906392 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.906393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:12.907230 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:12.907233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.907924 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.907925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:12.908006 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:12.908011 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 20:40:12.908064 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 20:40:12.908111 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 20:40:12.908210 12991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.905323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:54313 (local address 127.25.62.129:34353)
0619 20:40:12.905380 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:12.905382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:12.905396 (+ 14us) server_negotiation.cc:407] Connection header received
0619 20:40:12.905464 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:12.905466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:12.905476 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:12.905520 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:12.905764 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.906298 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.907345 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.907849 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.907906 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:12.907913 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:12.908080 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 20:40:12.908119 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 20:40:12.908315 13002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.905345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:35619 (local address 127.25.62.131:49399)
0619 20:40:12.905549 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:12.905558 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:12.905612 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:12.905756 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:12.905757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:12.905758 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:12.905831 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:12.905833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.906527 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.906527 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:12.907250 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:12.907251 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.908061 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.908062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:12.908131 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:12.908135 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:40:12.908193 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 20:40:12.908232 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":106,"threads_started":1}
I20260619 20:40:12.908336 12992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.905449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:49399 (local address 127.25.62.130:35619)
0619 20:40:12.905527 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:12.905529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:12.905624 (+ 95us) server_negotiation.cc:407] Connection header received
0619 20:40:12.905658 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:12.905659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:12.905670 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:12.905748 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:12.905928 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.906440 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.907391 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.907961 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.908033 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:12.908042 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:12.908213 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 20:40:12.908258 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 20:40:12.908722 12665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e06e49c7d6347cf9d4414472d0a0ee9" candidate_uuid: "6762781096b44bacb8e2beeacdf22fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09721578ee264c6ba2865a75376fa85a" is_pre_election: true
I20260619 20:40:12.908735 12799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e06e49c7d6347cf9d4414472d0a0ee9" candidate_uuid: "6762781096b44bacb8e2beeacdf22fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d677185faaac4e13973f9a201b469fbb" is_pre_election: true
I20260619 20:40:12.908881 12665 raft_consensus.cc:2454] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 09721578ee264c6ba2865a75376fa85a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6762781096b44bacb8e2beeacdf22fbf in term 0.
I20260619 20:40:12.908881 12799 raft_consensus.cc:2454] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6762781096b44bacb8e2beeacdf22fbf in term 0.
I20260619 20:40:12.909116 12867 leader_election.cc:302] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09721578ee264c6ba2865a75376fa85a, 6762781096b44bacb8e2beeacdf22fbf; no voters:
I20260619 20:40:12.909250 13000 raft_consensus.cc:2781] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:40:12.909302 13000 raft_consensus.cc:482] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:40:12.909337 13000 raft_consensus.cc:3037] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:12.910027 13000 raft_consensus.cc:504] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09721578ee264c6ba2865a75376fa85a" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34353 } } peers { permanent_uuid: "6762781096b44bacb8e2beeacdf22fbf" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 38275 } } peers { permanent_uuid: "d677185faaac4e13973f9a201b469fbb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35619 } }
I20260619 20:40:12.910152 13000 leader_election.cc:288] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [CANDIDATE]: Term 1 election: Requested vote from peers 09721578ee264c6ba2865a75376fa85a (127.25.62.129:34353), d677185faaac4e13973f9a201b469fbb (127.25.62.130:35619)
I20260619 20:40:12.910344 12799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e06e49c7d6347cf9d4414472d0a0ee9" candidate_uuid: "6762781096b44bacb8e2beeacdf22fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d677185faaac4e13973f9a201b469fbb"
I20260619 20:40:12.910362 12665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e06e49c7d6347cf9d4414472d0a0ee9" candidate_uuid: "6762781096b44bacb8e2beeacdf22fbf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09721578ee264c6ba2865a75376fa85a"
I20260619 20:40:12.910445 12799 raft_consensus.cc:3037] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:12.910441 12665 raft_consensus.cc:3037] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 09721578ee264c6ba2865a75376fa85a [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:12.911135 12665 raft_consensus.cc:2454] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 09721578ee264c6ba2865a75376fa85a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6762781096b44bacb8e2beeacdf22fbf in term 1.
I20260619 20:40:12.911248 12799 raft_consensus.cc:2454] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6762781096b44bacb8e2beeacdf22fbf in term 1.
I20260619 20:40:12.911293 12867 leader_election.cc:302] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09721578ee264c6ba2865a75376fa85a, 6762781096b44bacb8e2beeacdf22fbf; no voters:
I20260619 20:40:12.911384 13000 raft_consensus.cc:2781] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:12.911580 13000 raft_consensus.cc:686] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [term 1 LEADER]: Becoming Leader. State: Replica: 6762781096b44bacb8e2beeacdf22fbf, State: Running, Role: LEADER
I20260619 20:40:12.911689 13000 consensus_queue.cc:237] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09721578ee264c6ba2865a75376fa85a" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34353 } } peers { permanent_uuid: "6762781096b44bacb8e2beeacdf22fbf" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 38275 } } peers { permanent_uuid: "d677185faaac4e13973f9a201b469fbb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35619 } }
I20260619 20:40:12.912563 12520 catalog_manager.cc:5495] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf reported cstate change: term changed from 0 to 1, leader changed from <none> to 6762781096b44bacb8e2beeacdf22fbf (127.25.62.131). New cstate: current_term: 1 leader_uuid: "6762781096b44bacb8e2beeacdf22fbf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09721578ee264c6ba2865a75376fa85a" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6762781096b44bacb8e2beeacdf22fbf" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 38275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d677185faaac4e13973f9a201b469fbb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35619 } health_report { overall_health: UNKNOWN } } }
I20260619 20:40:12.940871 12989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.938335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40268 (local address 127.25.62.131:38275)
0619 20:40:12.938404 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:12.938405 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:12.938413 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:40:12.938517 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:12.938519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:12.938539 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:12.938622 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:40:12.938768 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.939241 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.940049 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:12.940260 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:12.940350 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:12.940520 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:40:12.940635 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:40:12.940730 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:12.940749 (+ 19us) server_negotiation.cc:299] Negotiation successful
0619 20:40:12.940767 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:40:12.942742 12799 raft_consensus.cc:1261] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb [term 1 FOLLOWER]: Refusing update from remote peer 6762781096b44bacb8e2beeacdf22fbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:40:12.942744 12665 raft_consensus.cc:1261] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 09721578ee264c6ba2865a75376fa85a [term 1 FOLLOWER]: Refusing update from remote peer 6762781096b44bacb8e2beeacdf22fbf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:40:12.943064 13003 consensus_queue.cc:1035] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "d677185faaac4e13973f9a201b469fbb" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 35619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:40:12.943192 13000 consensus_queue.cc:1035] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [LEADER]: Connected to new peer: Peer: permanent_uuid: "09721578ee264c6ba2865a75376fa85a" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 34353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:40:12.945950 13008 mvcc.cc:205] Tried to move back new op lower bound from 7298669006610731008 to 7298669006487035904. Current Snapshot: MvccSnapshot[applied={T|T < 7298669006610731008}]
I20260619 20:40:12.946453 13007 mvcc.cc:205] Tried to move back new op lower bound from 7298669006610731008 to 7298669006487035904. Current Snapshot: MvccSnapshot[applied={T|T < 7298669006610731008}]
I20260619 20:40:12.947777 12519 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:40:12.947861 12519 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:40:12.948580 12519 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 894f7b68c9da449a80fe3fe298fabce4: Sending DeleteTablet for 3 replicas of tablet 8e06e49c7d6347cf9d4414472d0a0ee9
I20260619 20:40:12.948954 12913 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e06e49c7d6347cf9d4414472d0a0ee9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:40:12 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:40256
I20260619 20:40:12.948971 12779 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e06e49c7d6347cf9d4414472d0a0ee9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:40:12 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:41396
I20260619 20:40:12.949129 12645 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e06e49c7d6347cf9d4414472d0a0ee9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:40:12 UTC) from {username='kudu', principal='kudu/127.25.62.190@KRBTEST.COM'} at 127.0.0.1:38784
I20260619 20:40:12.949390 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 12582
I20260619 20:40:12.949405 13016 tablet_replica.cc:330] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 09721578ee264c6ba2865a75376fa85a: stopping tablet replica
I20260619 20:40:12.951025 13017 tablet_replica.cc:330] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf: stopping tablet replica
I20260619 20:40:12.951195 13017 raft_consensus.cc:2229] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:40:12.951354 13017 raft_consensus.cc:2258] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:40:12.951949 13015 tablet_replica.cc:330] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb: stopping tablet replica
I20260619 20:40:12.952054 13015 raft_consensus.cc:2229] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:40:12.952158 13015 raft_consensus.cc:2258] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:40:12.952602 13015 ts_tablet_manager.cc:1905] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:40:12.953406 13017 ts_tablet_manager.cc:1905] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:40:12.954598 13015 ts_tablet_manager.cc:1918] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:40:12.954738 13015 log.cc:1196] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/8e06e49c7d6347cf9d4414472d0a0ee9
W20260619 20:40:12.955058 12505 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.62.129:34353 (error 108)
I20260619 20:40:12.955102 13015 ts_tablet_manager.cc:1939] T 8e06e49c7d6347cf9d4414472d0a0ee9 P d677185faaac4e13973f9a201b469fbb: Deleting consensus metadata
I20260619 20:40:12.955317 13017 ts_tablet_manager.cc:1918] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:40:12.955386 13017 log.cc:1196] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/8e06e49c7d6347cf9d4414472d0a0ee9
I20260619 20:40:12.955598 12990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.955454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:34353 (local address 127.0.0.1:38788)
0619 20:40:12.955521 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:12.955560 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.129:34353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 20:40:12.955741 13017 ts_tablet_manager.cc:1939] T 8e06e49c7d6347cf9d4414472d0a0ee9 P 6762781096b44bacb8e2beeacdf22fbf: Deleting consensus metadata
W20260619 20:40:12.955761 12505 catalog_manager.cc:4584] TS 09721578ee264c6ba2865a75376fa85a (127.25.62.129:34353): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8e06e49c7d6347cf9d4414472d0a0ee9: Network error: Client connection negotiation failed: client connection to 127.25.62.129:34353: connect: Connection refused (error 111)
I20260619 20:40:12.955961 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 12716
I20260619 20:40:12.957407 12504 catalog_manager.cc:4841] TS 6762781096b44bacb8e2beeacdf22fbf (127.25.62.131:38275): tablet 8e06e49c7d6347cf9d4414472d0a0ee9 (table test-table [id=af2b04e00760425c9bfaeb2158d1307b]) successfully deleted
I20260619 20:40:12.960577 12990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:12.960452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:35619 (local address 127.0.0.1:41400)
0619 20:40:12.960508 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:12.960538 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.130:35619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 20:40:12.961005 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 12850
I20260619 20:40:12.965245 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 12488
2026-06-19T20:40:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2001 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T20:40:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:12Z Disabled control of system clock
I20260619 20:40:12.992995 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:41637
--webserver_interface=127.25.62.190
--webserver_port=0
--builtin_ntp_servers=127.25.62.148:42379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:41637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:40:13.071044 13022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:13.071249 13022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:13.071269 13022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:13.072320 13022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:13.072352 13022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:13.072364 13022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:13.072377 13022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:13.072389 13022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:13.073632 13022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:42379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:41637
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:41637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:13.073819 13022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:13.074045 13022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:13.078663 13027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:13.078665 13030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:13.078780 13022 server_base.cc:1034] running on GCE node
W20260619 20:40:13.078701 13028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:13.079170 13022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:13.079420 13022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:13.080567 13022 hybrid_clock.cc:648] HybridClock initialized: now 1781901613080555 us; error 26 us; skew 500 ppm
I20260619 20:40:13.081868 13022 webserver.cc:460] Webserver started at http://127.25.62.190:33769/ using document root <none> and password file <none>
I20260619 20:40:13.082109 13022 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:13.082154 13022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:13.082304 13022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:13.083377 13022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "821a37c1bbb448d2b86a85d0bcc84a3c"
format_stamp: "Formatted at 2026-06-19 20:40:13 on dist-test-slave-0t40"
server_key: "ef0375fc575b194ae303e3115ff11d5f"
server_key_iv: "fbf6688e5bf874bcc7947471f1a222f9"
server_key_version: "encryptionkey@0"
I20260619 20:40:13.083742 13022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "821a37c1bbb448d2b86a85d0bcc84a3c"
format_stamp: "Formatted at 2026-06-19 20:40:13 on dist-test-slave-0t40"
server_key: "ef0375fc575b194ae303e3115ff11d5f"
server_key_iv: "fbf6688e5bf874bcc7947471f1a222f9"
server_key_version: "encryptionkey@0"
I20260619 20:40:13.085033 13022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:40:13.085839 13036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:13.086010 13022 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:40:13.086081 13022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "821a37c1bbb448d2b86a85d0bcc84a3c"
format_stamp: "Formatted at 2026-06-19 20:40:13 on dist-test-slave-0t40"
server_key: "ef0375fc575b194ae303e3115ff11d5f"
server_key_iv: "fbf6688e5bf874bcc7947471f1a222f9"
server_key_version: "encryptionkey@0"
I20260619 20:40:13.086172 13022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:13.114030 13022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:13.114362 13022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:13.114526 13022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:13.118539 13022 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:41637
I20260619 20:40:13.118594 13088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:41637 every 8 connection(s)
I20260619 20:40:13.118919 13022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:13.119948 13089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:13.122341 13089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c: Bootstrap starting.
I20260619 20:40:13.122942 13089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:13.123250 13089 log.cc:824] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:13.123948 13089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c: No bootstrap required, opened a new log
I20260619 20:40:13.125722 13089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821a37c1bbb448d2b86a85d0bcc84a3c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41637 } }
I20260619 20:40:13.125883 13089 raft_consensus.cc:374] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:13.125937 13089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 821a37c1bbb448d2b86a85d0bcc84a3c, State: Initialized, Role: FOLLOWER
I20260619 20:40:13.126139 13089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821a37c1bbb448d2b86a85d0bcc84a3c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41637 } }
I20260619 20:40:13.126247 13089 raft_consensus.cc:388] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:13.126294 13089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:13.126350 13089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:13.126881 13089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821a37c1bbb448d2b86a85d0bcc84a3c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41637 } }
I20260619 20:40:13.127015 13089 leader_election.cc:302] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 821a37c1bbb448d2b86a85d0bcc84a3c; no voters:
I20260619 20:40:13.127246 13089 leader_election.cc:288] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:13.127287 13092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:13.127342 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 13022
I20260619 20:40:13.127418 13092 raft_consensus.cc:686] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c [term 1 LEADER]: Becoming Leader. State: Replica: 821a37c1bbb448d2b86a85d0bcc84a3c, State: Running, Role: LEADER
I20260619 20:40:13.127453 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:13.127516 13089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:13.127503 13092 consensus_queue.cc:237] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821a37c1bbb448d2b86a85d0bcc84a3c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41637 } }
I20260619 20:40:13.127622 25850 external_mini_cluster.cc:1428] Setting key c5295fd67d713360c929c93b75db3775
I20260619 20:40:13.127841 13092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 821a37c1bbb448d2b86a85d0bcc84a3c. Latest consensus state: current_term: 1 leader_uuid: "821a37c1bbb448d2b86a85d0bcc84a3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821a37c1bbb448d2b86a85d0bcc84a3c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41637 } } }
I20260619 20:40:13.128019 13092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:13.128013 13093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "821a37c1bbb448d2b86a85d0bcc84a3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "821a37c1bbb448d2b86a85d0bcc84a3c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 41637 } } }
I20260619 20:40:13.128185 13093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:13.128499 13102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:13.129092 13102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:13.130836 13102 catalog_manager.cc:1347] Generated new cluster ID: 8a2e877b814344faaf7de3a54cf4405e
I20260619 20:40:13.130887 13102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:13.133620 13103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:13.128312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33012 (local address 127.25.62.190:41637)
0619 20:40:13.128810 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:13.128820 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:13.128871 (+ 51us) server_negotiation.cc:407] Connection header received
0619 20:40:13.128993 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:13.128998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:13.129094 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:13.129291 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:13.129514 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.130819 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.131606 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.132027 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.132168 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:13.132883 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:13.132886 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:13.132887 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:13.133122 (+ 235us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:13.133177 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:13.133188 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:13.133263 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:13.133338 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 20:40:13.133418 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":84,"threads_started":1}
I20260619 20:40:13.159037 13102 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:13.159755 13102 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:13.165966 13102 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c: Generated new TSK 0
I20260619 20:40:13.166172 13102 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:13.167585 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.129:0
--local_ip_for_outbound_sockets=127.25.62.129
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41637
--builtin_ntp_servers=127.25.62.148:42379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:40:13.249413 13113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:13.249620 13113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:13.249644 13113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:13.250833 13113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:13.250880 13113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:13.250909 13113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.129
I20260619 20:40:13.252332 13113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:42379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.25.62.129
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.129
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:13.252594 13113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:13.252936 13113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:13.258966 13119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:13.259068 13118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:13.258944 13121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:13.259310 13113 server_base.cc:1034] running on GCE node
I20260619 20:40:13.259527 13113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:13.259804 13113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:13.260993 13113 hybrid_clock.cc:648] HybridClock initialized: now 1781901613260975 us; error 43 us; skew 500 ppm
I20260619 20:40:13.262481 13113 webserver.cc:460] Webserver started at http://127.25.62.129:33667/ using document root <none> and password file <none>
I20260619 20:40:13.262749 13113 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:13.262804 13113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:13.262925 13113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:13.264214 13113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-0/data/instance:
uuid: "87cd5e4253a149a4b7732ad3631ad145"
format_stamp: "Formatted at 2026-06-19 20:40:13 on dist-test-slave-0t40"
server_key: "45fe5310a1d3d9aa661e5cdc7222c7f1"
server_key_iv: "2e5b3671279da318cd0402b2b7186be5"
server_key_version: "encryptionkey@0"
I20260619 20:40:13.264590 13113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance:
uuid: "87cd5e4253a149a4b7732ad3631ad145"
format_stamp: "Formatted at 2026-06-19 20:40:13 on dist-test-slave-0t40"
server_key: "45fe5310a1d3d9aa661e5cdc7222c7f1"
server_key_iv: "2e5b3671279da318cd0402b2b7186be5"
server_key_version: "encryptionkey@0"
I20260619 20:40:13.266243 13113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260619 20:40:13.267066 13127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:13.267248 13113 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:40:13.267355 13113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-0/wal
uuid: "87cd5e4253a149a4b7732ad3631ad145"
format_stamp: "Formatted at 2026-06-19 20:40:13 on dist-test-slave-0t40"
server_key: "45fe5310a1d3d9aa661e5cdc7222c7f1"
server_key_iv: "2e5b3671279da318cd0402b2b7186be5"
server_key_version: "encryptionkey@0"
I20260619 20:40:13.267428 13113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:13.295521 13113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:13.295842 13113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:13.295997 13113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:13.296233 13113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:40:13.296581 13113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:40:13.296617 13113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:13.296638 13113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:40:13.296684 13113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:13.302310 13113 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.129:38523
I20260619 20:40:13.302423 13240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.129:38523 every 8 connection(s)
I20260619 20:40:13.302738 13113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-0/data/info.pb
I20260619 20:40:13.307933 13103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:13.303497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.129:41801 (local address 127.25.62.190:41637)
0619 20:40:13.303562 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:13.303564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:13.303809 (+ 245us) server_negotiation.cc:407] Connection header received
0619 20:40:13.304330 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:13.304331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:13.304345 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:13.304401 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:13.305075 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.305597 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.306576 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.306803 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.306883 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:13.307656 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:13.307657 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:13.307657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:13.307662 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:13.307677 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:13.307681 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:13.307735 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:13.307824 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 20:40:13.307855 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 20:40:13.308161 13243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:13.303487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:41637 (local address 127.25.62.129:41801)
0619 20:40:13.303694 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:13.303727 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:13.304223 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:13.304472 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:13.304476 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:13.304484 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:13.304965 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:13.304969 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.305708 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.305708 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:13.306484 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:13.306486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.306890 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.306891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:13.307018 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:13.307508 (+ 490us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:40:13.307508 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:13.307537 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:40:13.307558 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:13.307738 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:13.307742 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:13.307745 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:40:13.307829 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 20:40:13.307925 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":108,"threads_started":1}
I20260619 20:40:13.308640 13241 heartbeater.cc:347] Connected to a master server at 127.25.62.190:41637
I20260619 20:40:13.308743 13241 heartbeater.cc:464] Registering TS with master...
I20260619 20:40:13.308933 13241 heartbeater.cc:511] Master 127.25.62.190:41637 requested a full tablet report, sending...
I20260619 20:40:13.309473 13053 ts_manager.cc:194] Registered new tserver with Master: 87cd5e4253a149a4b7732ad3631ad145 (127.25.62.129:38523)
I20260619 20:40:13.310014 13053 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.62.129:41801
I20260619 20:40:13.312704 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 13113
I20260619 20:40:13.312824 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-0/wal/instance
I20260619 20:40:13.313007 25850 external_mini_cluster.cc:1428] Setting key 6fd4793a8bf9f3804c3476f65808eddb
I20260619 20:40:13.313871 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.130:0
--local_ip_for_outbound_sockets=127.25.62.130
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41637
--builtin_ntp_servers=127.25.62.148:42379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 20:40:13.392501 13244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:13.392690 13244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:13.392710 13244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:13.393774 13244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:13.393812 13244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:13.393836 13244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.130
I20260619 20:40:13.395067 13244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:42379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.25.62.130
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.130
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:13.395237 13244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:13.395449 13244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:13.400213 13252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:13.400204 13249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:13.400244 13250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:13.400487 13244 server_base.cc:1034] running on GCE node
I20260619 20:40:13.400681 13244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:13.400916 13244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:13.402098 13244 hybrid_clock.cc:648] HybridClock initialized: now 1781901613402076 us; error 32 us; skew 500 ppm
I20260619 20:40:13.403326 13244 webserver.cc:460] Webserver started at http://127.25.62.130:36389/ using document root <none> and password file <none>
I20260619 20:40:13.403553 13244 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:13.403617 13244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:13.403761 13244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:13.404789 13244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-1/data/instance:
uuid: "451a59792d72462ebcbc1c41ecf7be83"
format_stamp: "Formatted at 2026-06-19 20:40:13 on dist-test-slave-0t40"
server_key: "049ea9e08006978c50e4192b1857e0bd"
server_key_iv: "7f35bd9de2381478a2a88f06957c58e1"
server_key_version: "encryptionkey@0"
I20260619 20:40:13.405187 13244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance:
uuid: "451a59792d72462ebcbc1c41ecf7be83"
format_stamp: "Formatted at 2026-06-19 20:40:13 on dist-test-slave-0t40"
server_key: "049ea9e08006978c50e4192b1857e0bd"
server_key_iv: "7f35bd9de2381478a2a88f06957c58e1"
server_key_version: "encryptionkey@0"
I20260619 20:40:13.406477 13244 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:40:13.407178 13258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:13.407347 13244 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:40:13.407444 13244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-1/wal
uuid: "451a59792d72462ebcbc1c41ecf7be83"
format_stamp: "Formatted at 2026-06-19 20:40:13 on dist-test-slave-0t40"
server_key: "049ea9e08006978c50e4192b1857e0bd"
server_key_iv: "7f35bd9de2381478a2a88f06957c58e1"
server_key_version: "encryptionkey@0"
I20260619 20:40:13.407524 13244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:13.420166 13244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:13.420488 13244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:13.420627 13244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:13.420857 13244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:40:13.421262 13244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:40:13.421314 13244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:13.421350 13244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:40:13.421376 13244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:13.427206 13244 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.130:45067
I20260619 20:40:13.427271 13371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.130:45067 every 8 connection(s)
I20260619 20:40:13.427636 13244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-1/data/info.pb
I20260619 20:40:13.428289 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 13244
I20260619 20:40:13.428395 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-1/wal/instance
I20260619 20:40:13.428592 25850 external_mini_cluster.cc:1428] Setting key 2eb483caaa2cbda67ace3301327dca97
I20260619 20:40:13.429695 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.25.62.131:0
--local_ip_for_outbound_sockets=127.25.62.131
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41637
--builtin_ntp_servers=127.25.62.148:42379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 20:40:13.434038 13103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:13.428416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:53477 (local address 127.25.62.190:41637)
0619 20:40:13.428466 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:13.428468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:13.428741 (+ 273us) server_negotiation.cc:407] Connection header received
0619 20:40:13.429433 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:13.429434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:13.429446 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:13.429512 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:13.430233 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.431000 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.432510 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.432693 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.432749 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:13.433765 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:13.433766 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:13.433766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:13.433771 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:13.433786 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:13.433790 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:13.433838 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:13.433947 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 20:40:13.433980 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:40:13.434341 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:13.428432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:41637 (local address 127.25.62.130:53477)
0619 20:40:13.428605 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:13.428646 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:13.429326 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:13.429563 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:13.429566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:13.429574 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:13.430137 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:13.430143 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.431103 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.431104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:13.432424 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:13.432427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.432776 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.432777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:13.432938 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:13.433600 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:40:13.433600 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:13.433644 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:40:13.433677 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:13.433858 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:13.433864 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:13.433867 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:40:13.433945 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 20:40:13.434064 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260619 20:40:13.434850 13372 heartbeater.cc:347] Connected to a master server at 127.25.62.190:41637
I20260619 20:40:13.434957 13372 heartbeater.cc:464] Registering TS with master...
I20260619 20:40:13.435174 13372 heartbeater.cc:511] Master 127.25.62.190:41637 requested a full tablet report, sending...
I20260619 20:40:13.435552 13053 ts_manager.cc:194] Registered new tserver with Master: 451a59792d72462ebcbc1c41ecf7be83 (127.25.62.130:45067)
I20260619 20:40:13.435912 13053 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.62.130:53477
W20260619 20:40:13.511169 13375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:13.511363 13375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:13.511384 13375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:13.512514 13375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:13.512554 13375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:13.512580 13375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.25.62.131
I20260619 20:40:13.513911 13375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:42379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.25.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.25.62.131
--webserver_port=0
--tserver_master_addrs=127.25.62.190:41637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.25.62.131
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:13.514174 13375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:13.514474 13375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:13.519331 13381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:13.519375 13380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:13.519364 13383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:13.519482 13375 server_base.cc:1034] running on GCE node
I20260619 20:40:13.519985 13375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:13.520180 13375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:13.521320 13375 hybrid_clock.cc:648] HybridClock initialized: now 1781901613521235 us; error 102 us; skew 500 ppm
I20260619 20:40:13.522522 13375 webserver.cc:460] Webserver started at http://127.25.62.131:43001/ using document root <none> and password file <none>
I20260619 20:40:13.522778 13375 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:13.522848 13375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:13.522995 13375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:13.524015 13375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-2/data/instance:
uuid: "6027aa5172f84d7291643595b70593db"
format_stamp: "Formatted at 2026-06-19 20:40:13 on dist-test-slave-0t40"
server_key: "28595a454a60a29906177c4ca06382ef"
server_key_iv: "d04a2814d7ebc289d4a60e824b3f3449"
server_key_version: "encryptionkey@0"
I20260619 20:40:13.524386 13375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance:
uuid: "6027aa5172f84d7291643595b70593db"
format_stamp: "Formatted at 2026-06-19 20:40:13 on dist-test-slave-0t40"
server_key: "28595a454a60a29906177c4ca06382ef"
server_key_iv: "d04a2814d7ebc289d4a60e824b3f3449"
server_key_version: "encryptionkey@0"
I20260619 20:40:13.525703 13375 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:40:13.526402 13389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:13.526576 13375 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:40:13.526671 13375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-2/wal
uuid: "6027aa5172f84d7291643595b70593db"
format_stamp: "Formatted at 2026-06-19 20:40:13 on dist-test-slave-0t40"
server_key: "28595a454a60a29906177c4ca06382ef"
server_key_iv: "d04a2814d7ebc289d4a60e824b3f3449"
server_key_version: "encryptionkey@0"
I20260619 20:40:13.526753 13375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:13.559922 13375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:13.560262 13375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:13.560418 13375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:13.560652 13375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 20:40:13.560987 13375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 20:40:13.561043 13375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:13.561118 13375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 20:40:13.561143 13375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:13.566916 13375 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.131:41215
I20260619 20:40:13.566968 13502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.131:41215 every 8 connection(s)
I20260619 20:40:13.567332 13375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-2/data/info.pb
I20260619 20:40:13.572423 13103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:13.568064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.131:34327 (local address 127.25.62.190:41637)
0619 20:40:13.568152 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:13.568154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:13.568365 (+ 211us) server_negotiation.cc:407] Connection header received
0619 20:40:13.568829 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:13.568830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:13.568840 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:13.568891 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:13.569591 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.570146 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.571128 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.571359 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.571422 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:13.572217 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:13.572218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:13.572218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:13.572222 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:13.572235 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:13.572239 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:13.572281 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:13.572341 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 20:40:13.572364 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 20:40:13.572654 13505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:13.568028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:41637 (local address 127.25.62.131:34327)
0619 20:40:13.568241 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:13.568275 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:13.568738 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:13.568944 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:13.568947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:13.568954 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:13.569495 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:13.569500 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.570270 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.570271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:13.571046 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:13.571048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.571457 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.571458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:13.571581 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:13.572079 (+ 498us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 20:40:13.572080 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:13.572111 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 20:40:13.572133 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:13.572281 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:13.572283 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:13.572285 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 20:40:13.572332 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 20:40:13.572406 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":108,"threads_started":1}
I20260619 20:40:13.573195 13503 heartbeater.cc:347] Connected to a master server at 127.25.62.190:41637
I20260619 20:40:13.573287 13503 heartbeater.cc:464] Registering TS with master...
I20260619 20:40:13.573424 13503 heartbeater.cc:511] Master 127.25.62.190:41637 requested a full tablet report, sending...
I20260619 20:40:13.573731 13053 ts_manager.cc:194] Registered new tserver with Master: 6027aa5172f84d7291643595b70593db (127.25.62.131:41215)
I20260619 20:40:13.574051 13053 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.25.62.131:34327
I20260619 20:40:13.575373 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 13375
I20260619 20:40:13.575459 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-2/wal/instance
I20260619 20:40:13.575598 25850 external_mini_cluster.cc:1428] Setting key 0273706f604a88b32c3d56668a49a8c5
I20260619 20:40:13.576471 25850 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 20:40:13.580417 13103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:13.577802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33022 (local address 127.25.62.190:41637)
0619 20:40:13.577872 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:13.577874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:13.577982 (+ 108us) server_negotiation.cc:407] Connection header received
0619 20:40:13.578001 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:13.578002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:13.578015 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:13.578059 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:13.578219 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.578667 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.579370 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.579543 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.579600 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:13.580167 (+ 567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:13.580169 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:13.580169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:13.580173 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:13.580186 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:13.580189 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:13.580240 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:13.580283 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 20:40:13.580305 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 20:40:13.581892 13053 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 20:40:13.583405 13053 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 20:40:13.589488 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:13.585536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.130:45067 (local address 127.0.0.1:57222)
0619 20:40:13.585680 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:13.585716 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:13.585776 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:13.586257 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:13.586259 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:13.586261 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:13.586330 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:13.586332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.587303 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.587304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:13.588230 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:13.588232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.589179 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.589180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:13.589260 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:13.589274 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 20:40:13.589362 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 20:40:13.589413 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":47,"threads_started":1}
I20260619 20:40:13.589493 13516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:13.585939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57222 (local address 127.25.62.130:45067)
0619 20:40:13.586100 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:13.586103 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:13.586111 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:40:13.586141 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:13.586142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:13.586153 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:13.586238 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:13.586420 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.587221 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.588335 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.589086 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.589175 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:13.589263 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:13.589374 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 20:40:13.589410 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260619 20:40:13.590199 13511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:13.585536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:38523 (local address 127.0.0.1:49032)
0619 20:40:13.585919 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:13.585930 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:13.585976 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:13.586502 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:13.586503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:13.586505 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:13.586560 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:13.586562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.587926 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.587927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:13.588994 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:13.588997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.589943 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.589944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:13.590007 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:13.590011 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:40:13.590083 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 20:40:13.590130 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":3,"thread_start_us":74,"threads_started":1}
I20260619 20:40:13.590209 13514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:13.585684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49032 (local address 127.25.62.129:38523)
0619 20:40:13.586121 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:13.586125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:13.586135 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:40:13.586169 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:13.586170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:13.586183 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:13.586650 (+ 467us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:13.586765 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.587838 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.589125 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.589856 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.589922 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:13.590002 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:13.590094 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 20:40:13.590129 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":142,"threads_started":1}
I20260619 20:40:13.590247 13306 tablet_service.cc:1467] Processing CreateTablet for tablet ca77d1aae200413196102400c19f491a (DEFAULT_TABLE table=test-table [id=7034aa2266d5408aa8c4b7f42158339a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:40:13.590638 13306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca77d1aae200413196102400c19f491a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:13.590652 13513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:13.585718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:41215 (local address 127.0.0.1:40868)
0619 20:40:13.586339 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:13.586349 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:13.586403 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:13.586602 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:13.586603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:13.586605 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:13.586641 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:13.586643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.587943 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.587944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:13.588983 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:13.588988 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.590394 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.590395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:13.590439 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:13.590442 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 20:40:13.590503 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 20:40:13.590557 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":93,"threads_started":1}
I20260619 20:40:13.590924 13515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:13.585826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40868 (local address 127.25.62.131:41215)
0619 20:40:13.586002 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:13.586007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:13.586423 (+ 416us) server_negotiation.cc:407] Connection header received
0619 20:40:13.586464 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:13.586466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:13.586479 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:13.586548 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:13.586733 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.587833 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.589123 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.590086 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.590150 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:13.590768 (+ 618us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:13.590801 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 20:40:13.590844 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":81,"threads_started":1}
I20260619 20:40:13.591467 13437 tablet_service.cc:1467] Processing CreateTablet for tablet ca77d1aae200413196102400c19f491a (DEFAULT_TABLE table=test-table [id=7034aa2266d5408aa8c4b7f42158339a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:40:13.591714 13437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca77d1aae200413196102400c19f491a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:13.593525 13517 tablet_bootstrap.cc:492] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83: Bootstrap starting.
I20260619 20:40:13.593755 13175 tablet_service.cc:1467] Processing CreateTablet for tablet ca77d1aae200413196102400c19f491a (DEFAULT_TABLE table=test-table [id=7034aa2266d5408aa8c4b7f42158339a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 20:40:13.594084 13175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca77d1aae200413196102400c19f491a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:13.594240 13517 tablet_bootstrap.cc:654] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:13.594565 13517 log.cc:824] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:13.595293 13517 tablet_bootstrap.cc:492] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83: No bootstrap required, opened a new log
I20260619 20:40:13.595362 13517 ts_tablet_manager.cc:1397] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:40:13.597373 13520 tablet_bootstrap.cc:492] T ca77d1aae200413196102400c19f491a P 87cd5e4253a149a4b7732ad3631ad145: Bootstrap starting.
I20260619 20:40:13.597451 13518 tablet_bootstrap.cc:492] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db: Bootstrap starting.
I20260619 20:40:13.597927 13520 tablet_bootstrap.cc:654] T ca77d1aae200413196102400c19f491a P 87cd5e4253a149a4b7732ad3631ad145: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:13.598059 13518 tablet_bootstrap.cc:654] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:13.597972 13517 raft_consensus.cc:348] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "451a59792d72462ebcbc1c41ecf7be83" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 45067 } } peers { permanent_uuid: "87cd5e4253a149a4b7732ad3631ad145" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38523 } } peers { permanent_uuid: "6027aa5172f84d7291643595b70593db" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 41215 } }
I20260619 20:40:13.598124 13517 raft_consensus.cc:374] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:13.598171 13517 raft_consensus.cc:729] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 451a59792d72462ebcbc1c41ecf7be83, State: Initialized, Role: FOLLOWER
I20260619 20:40:13.598278 13520 log.cc:824] T ca77d1aae200413196102400c19f491a P 87cd5e4253a149a4b7732ad3631ad145: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:13.598291 13517 consensus_queue.cc:260] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "451a59792d72462ebcbc1c41ecf7be83" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 45067 } } peers { permanent_uuid: "87cd5e4253a149a4b7732ad3631ad145" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38523 } } peers { permanent_uuid: "6027aa5172f84d7291643595b70593db" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 41215 } }
I20260619 20:40:13.598374 13518 log.cc:824] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:13.598548 13517 ts_tablet_manager.cc:1428] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 20:40:13.598683 13372 heartbeater.cc:503] Master 127.25.62.190:41637 was elected leader, sending a full tablet report...
I20260619 20:40:13.599000 13520 tablet_bootstrap.cc:492] T ca77d1aae200413196102400c19f491a P 87cd5e4253a149a4b7732ad3631ad145: No bootstrap required, opened a new log
I20260619 20:40:13.599066 13520 ts_tablet_manager.cc:1397] T ca77d1aae200413196102400c19f491a P 87cd5e4253a149a4b7732ad3631ad145: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 20:40:13.600849 13520 raft_consensus.cc:348] T ca77d1aae200413196102400c19f491a P 87cd5e4253a149a4b7732ad3631ad145 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "451a59792d72462ebcbc1c41ecf7be83" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 45067 } } peers { permanent_uuid: "87cd5e4253a149a4b7732ad3631ad145" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38523 } } peers { permanent_uuid: "6027aa5172f84d7291643595b70593db" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 41215 } }
I20260619 20:40:13.600994 13520 raft_consensus.cc:374] T ca77d1aae200413196102400c19f491a P 87cd5e4253a149a4b7732ad3631ad145 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:13.601060 13520 raft_consensus.cc:729] T ca77d1aae200413196102400c19f491a P 87cd5e4253a149a4b7732ad3631ad145 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87cd5e4253a149a4b7732ad3631ad145, State: Initialized, Role: FOLLOWER
I20260619 20:40:13.601181 13520 consensus_queue.cc:260] T ca77d1aae200413196102400c19f491a P 87cd5e4253a149a4b7732ad3631ad145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "451a59792d72462ebcbc1c41ecf7be83" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 45067 } } peers { permanent_uuid: "87cd5e4253a149a4b7732ad3631ad145" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38523 } } peers { permanent_uuid: "6027aa5172f84d7291643595b70593db" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 41215 } }
I20260619 20:40:13.601549 13520 ts_tablet_manager.cc:1428] T ca77d1aae200413196102400c19f491a P 87cd5e4253a149a4b7732ad3631ad145: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 20:40:13.601668 13241 heartbeater.cc:503] Master 127.25.62.190:41637 was elected leader, sending a full tablet report...
I20260619 20:40:13.601996 13518 tablet_bootstrap.cc:492] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db: No bootstrap required, opened a new log
I20260619 20:40:13.602082 13518 ts_tablet_manager.cc:1397] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260619 20:40:13.603864 13518 raft_consensus.cc:348] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "451a59792d72462ebcbc1c41ecf7be83" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 45067 } } peers { permanent_uuid: "87cd5e4253a149a4b7732ad3631ad145" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38523 } } peers { permanent_uuid: "6027aa5172f84d7291643595b70593db" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 41215 } }
I20260619 20:40:13.604001 13518 raft_consensus.cc:374] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:13.604044 13518 raft_consensus.cc:729] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6027aa5172f84d7291643595b70593db, State: Initialized, Role: FOLLOWER
I20260619 20:40:13.604149 13518 consensus_queue.cc:260] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "451a59792d72462ebcbc1c41ecf7be83" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 45067 } } peers { permanent_uuid: "87cd5e4253a149a4b7732ad3631ad145" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38523 } } peers { permanent_uuid: "6027aa5172f84d7291643595b70593db" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 41215 } }
I20260619 20:40:13.604391 13518 ts_tablet_manager.cc:1428] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 20:40:13.604449 13503 heartbeater.cc:503] Master 127.25.62.190:41637 was elected leader, sending a full tablet report...
W20260619 20:40:13.678268 13373 tablet.cc:2369] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:40:13.803491 13242 tablet.cc:2369] T ca77d1aae200413196102400c19f491a P 87cd5e4253a149a4b7732ad3631ad145: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 20:40:13.817902 13504 tablet.cc:2369] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 20:40:13.950461 13521 raft_consensus.cc:482] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 20:40:13.950645 13521 raft_consensus.cc:504] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "451a59792d72462ebcbc1c41ecf7be83" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 45067 } } peers { permanent_uuid: "87cd5e4253a149a4b7732ad3631ad145" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38523 } } peers { permanent_uuid: "6027aa5172f84d7291643595b70593db" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 41215 } }
I20260619 20:40:13.951073 13521 leader_election.cc:288] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87cd5e4253a149a4b7732ad3631ad145 (127.25.62.129:38523), 6027aa5172f84d7291643595b70593db (127.25.62.131:41215)
I20260619 20:40:13.954098 13515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:13.951351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:38097 (local address 127.25.62.131:41215)
0619 20:40:13.951401 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:13.951404 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:13.951495 (+ 91us) server_negotiation.cc:407] Connection header received
0619 20:40:13.951589 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:13.951589 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:13.951599 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:13.951674 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:13.951827 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.952318 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.953386 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.953751 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.953827 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:13.953835 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:13.953984 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 20:40:13.954022 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:40:13.954123 13514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:13.951425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.25.62.130:55105 (local address 127.25.62.129:38523)
0619 20:40:13.951497 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:13.951499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:13.951512 (+ 13us) server_negotiation.cc:407] Connection header received
0619 20:40:13.951598 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:13.951603 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:13.951614 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:13.951680 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 20:40:13.951876 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.952328 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.953400 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.953749 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.953845 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:13.953853 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:13.954016 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 20:40:13.954056 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 20:40:13.954114 13526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:13.951268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.131:41215 (local address 127.25.62.130:38097)
0619 20:40:13.951404 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:13.951414 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:13.951505 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:13.951674 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:13.951675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:13.951678 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:13.951747 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:13.951749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.952403 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.952404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:13.953294 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:13.953296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.953836 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.953837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:13.953897 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:13.953901 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:40:13.953955 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 20:40:13.954015 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":31,"thread_start_us":73,"threads_started":1}
I20260619 20:40:13.954119 13527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:13.951287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:38523 (local address 127.25.62.130:55105)
0619 20:40:13.951457 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:13.951467 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:13.951524 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:13.951695 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:13.951696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:13.951698 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 20:40:13.951776 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:13.951778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.952427 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.952428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:13.953293 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:13.953295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:13.953856 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:13.953856 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:13.953918 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:13.953922 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 20:40:13.953996 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 20:40:13.954030 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":32,"thread_start_us":61,"threads_started":1}
I20260619 20:40:13.954432 13457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca77d1aae200413196102400c19f491a" candidate_uuid: "451a59792d72462ebcbc1c41ecf7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6027aa5172f84d7291643595b70593db" is_pre_election: true
I20260619 20:40:13.954610 13457 raft_consensus.cc:2454] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 451a59792d72462ebcbc1c41ecf7be83 in term 0.
I20260619 20:40:13.954679 13195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca77d1aae200413196102400c19f491a" candidate_uuid: "451a59792d72462ebcbc1c41ecf7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87cd5e4253a149a4b7732ad3631ad145" is_pre_election: true
I20260619 20:40:13.954856 13195 raft_consensus.cc:2454] T ca77d1aae200413196102400c19f491a P 87cd5e4253a149a4b7732ad3631ad145 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 451a59792d72462ebcbc1c41ecf7be83 in term 0.
I20260619 20:40:13.954893 13260 leader_election.cc:302] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 451a59792d72462ebcbc1c41ecf7be83, 6027aa5172f84d7291643595b70593db; no voters:
I20260619 20:40:13.955014 13521 raft_consensus.cc:2781] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 20:40:13.955089 13521 raft_consensus.cc:482] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 20:40:13.955132 13521 raft_consensus.cc:3037] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:13.955845 13521 raft_consensus.cc:504] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "451a59792d72462ebcbc1c41ecf7be83" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 45067 } } peers { permanent_uuid: "87cd5e4253a149a4b7732ad3631ad145" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38523 } } peers { permanent_uuid: "6027aa5172f84d7291643595b70593db" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 41215 } }
I20260619 20:40:13.955998 13521 leader_election.cc:288] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [CANDIDATE]: Term 1 election: Requested vote from peers 87cd5e4253a149a4b7732ad3631ad145 (127.25.62.129:38523), 6027aa5172f84d7291643595b70593db (127.25.62.131:41215)
I20260619 20:40:13.956198 13195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca77d1aae200413196102400c19f491a" candidate_uuid: "451a59792d72462ebcbc1c41ecf7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87cd5e4253a149a4b7732ad3631ad145"
I20260619 20:40:13.956198 13457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca77d1aae200413196102400c19f491a" candidate_uuid: "451a59792d72462ebcbc1c41ecf7be83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6027aa5172f84d7291643595b70593db"
I20260619 20:40:13.956279 13195 raft_consensus.cc:3037] T ca77d1aae200413196102400c19f491a P 87cd5e4253a149a4b7732ad3631ad145 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:13.956279 13457 raft_consensus.cc:3037] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:13.957214 13457 raft_consensus.cc:2454] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 451a59792d72462ebcbc1c41ecf7be83 in term 1.
I20260619 20:40:13.957214 13195 raft_consensus.cc:2454] T ca77d1aae200413196102400c19f491a P 87cd5e4253a149a4b7732ad3631ad145 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 451a59792d72462ebcbc1c41ecf7be83 in term 1.
I20260619 20:40:13.957425 13261 leader_election.cc:302] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 451a59792d72462ebcbc1c41ecf7be83, 87cd5e4253a149a4b7732ad3631ad145; no voters:
I20260619 20:40:13.957521 13521 raft_consensus.cc:2781] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:13.957650 13521 raft_consensus.cc:686] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [term 1 LEADER]: Becoming Leader. State: Replica: 451a59792d72462ebcbc1c41ecf7be83, State: Running, Role: LEADER
I20260619 20:40:13.957778 13521 consensus_queue.cc:237] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "451a59792d72462ebcbc1c41ecf7be83" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 45067 } } peers { permanent_uuid: "87cd5e4253a149a4b7732ad3631ad145" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38523 } } peers { permanent_uuid: "6027aa5172f84d7291643595b70593db" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 41215 } }
I20260619 20:40:13.958667 13053 catalog_manager.cc:5495] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 reported cstate change: term changed from 0 to 1, leader changed from <none> to 451a59792d72462ebcbc1c41ecf7be83 (127.25.62.130). New cstate: current_term: 1 leader_uuid: "451a59792d72462ebcbc1c41ecf7be83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "451a59792d72462ebcbc1c41ecf7be83" member_type: VOTER last_known_addr { host: "127.25.62.130" port: 45067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87cd5e4253a149a4b7732ad3631ad145" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6027aa5172f84d7291643595b70593db" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 41215 } health_report { overall_health: UNKNOWN } } }
I20260619 20:40:14.029093 13516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:14.026612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57228 (local address 127.25.62.130:45067)
0619 20:40:14.026676 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:14.026678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:14.026686 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:40:14.026727 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:14.026728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:14.026754 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:14.026839 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:40:14.026976 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:14.027474 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:14.028291 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:14.028501 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:14.028577 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:14.028744 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:40:14.028853 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:40:14.028946 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:14.028965 (+ 19us) server_negotiation.cc:299] Negotiation successful
0619 20:40:14.028983 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 20:40:14.030951 13457 raft_consensus.cc:1261] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db [term 1 FOLLOWER]: Refusing update from remote peer 451a59792d72462ebcbc1c41ecf7be83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:40:14.031020 13195 raft_consensus.cc:1261] T ca77d1aae200413196102400c19f491a P 87cd5e4253a149a4b7732ad3631ad145 [term 1 FOLLOWER]: Refusing update from remote peer 451a59792d72462ebcbc1c41ecf7be83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:40:14.031394 13528 consensus_queue.cc:1035] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87cd5e4253a149a4b7732ad3631ad145" member_type: VOTER last_known_addr { host: "127.25.62.129" port: 38523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:40:14.031544 13521 consensus_queue.cc:1035] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6027aa5172f84d7291643595b70593db" member_type: VOTER last_known_addr { host: "127.25.62.131" port: 41215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:40:14.033864 13531 mvcc.cc:205] Tried to move back new op lower bound from 7298669011067883520 to 7298669010771742720. Current Snapshot: MvccSnapshot[applied={T|T < 7298669011067883520}]
I20260619 20:40:14.034327 13533 mvcc.cc:205] Tried to move back new op lower bound from 7298669011067883520 to 7298669010771742720. Current Snapshot: MvccSnapshot[applied={T|T < 7298669011067883520}]
I20260619 20:40:14.035790 13532 mvcc.cc:205] Tried to move back new op lower bound from 7298669011067883520 to 7298669010771742720. Current Snapshot: MvccSnapshot[applied={T|T < 7298669011067883520}]
I20260619 20:40:14.035880 13052 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:40:14.035969 13052 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 20:40:14.036715 13052 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 821a37c1bbb448d2b86a85d0bcc84a3c: Sending DeleteTablet for 3 replicas of tablet ca77d1aae200413196102400c19f491a
I20260619 20:40:14.037099 13175 tablet_service.cc:1514] Processing DeleteTablet for tablet ca77d1aae200413196102400c19f491a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:40:14 UTC) from {username='slave'} at 127.0.0.1:49032
I20260619 20:40:14.037103 13437 tablet_service.cc:1514] Processing DeleteTablet for tablet ca77d1aae200413196102400c19f491a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:40:14 UTC) from {username='slave'} at 127.0.0.1:40868
I20260619 20:40:14.037274 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 13113
I20260619 20:40:14.037353 13306 tablet_service.cc:1514] Processing DeleteTablet for tablet ca77d1aae200413196102400c19f491a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 20:40:14 UTC) from {username='slave'} at 127.0.0.1:57222
I20260619 20:40:14.037550 13543 tablet_replica.cc:330] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83: stopping tablet replica
I20260619 20:40:14.037626 13543 raft_consensus.cc:2229] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [term 1 LEADER]: Raft consensus shutting down.
I20260619 20:40:14.037726 13543 raft_consensus.cc:2258] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:40:14.037879 13541 tablet_replica.cc:330] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db: stopping tablet replica
I20260619 20:40:14.037961 13541 raft_consensus.cc:2229] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 20:40:14.038048 13541 raft_consensus.cc:2258] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 20:40:14.038345 13543 ts_tablet_manager.cc:1905] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:40:14.040395 13543 ts_tablet_manager.cc:1918] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:40:14.040586 13543 log.cc:1196] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-1/wal/wals/ca77d1aae200413196102400c19f491a
I20260619 20:40:14.040934 13543 ts_tablet_manager.cc:1939] T ca77d1aae200413196102400c19f491a P 451a59792d72462ebcbc1c41ecf7be83: Deleting consensus metadata
I20260619 20:40:14.041002 13541 ts_tablet_manager.cc:1905] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 20:40:14.041468 13038 catalog_manager.cc:4841] TS 451a59792d72462ebcbc1c41ecf7be83 (127.25.62.130:45067): tablet ca77d1aae200413196102400c19f491a (table test-table [id=7034aa2266d5408aa8c4b7f42158339a]) successfully deleted
I20260619 20:40:14.042590 13541 ts_tablet_manager.cc:1918] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 20:40:14.042663 13541 log.cc:1196] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db: Deleting WAL directory at /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781901378226483-25850-0/minicluster-data/ts-2/wal/wals/ca77d1aae200413196102400c19f491a
I20260619 20:40:14.042893 13541 ts_tablet_manager.cc:1939] T ca77d1aae200413196102400c19f491a P 6027aa5172f84d7291643595b70593db: Deleting consensus metadata
I20260619 20:40:14.043301 13038 catalog_manager.cc:4841] TS 6027aa5172f84d7291643595b70593db (127.25.62.131:41215): tablet ca77d1aae200413196102400c19f491a (table test-table [id=7034aa2266d5408aa8c4b7f42158339a]) successfully deleted
W20260619 20:40:14.044451 13039 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.25.62.129:38523 (error 108)
I20260619 20:40:14.044648 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 13244
I20260619 20:40:14.045235 13513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:14.044773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.129:38523 (local address 127.0.0.1:49040)
0619 20:40:14.045154 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:14.045196 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.129:38523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361}
W20260619 20:40:14.045579 13039 catalog_manager.cc:4584] TS 87cd5e4253a149a4b7732ad3631ad145 (127.25.62.129:38523): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ca77d1aae200413196102400c19f491a: Network error: Client connection negotiation failed: client connection to 127.25.62.129:38523: connect: Connection refused (error 111)
I20260619 20:40:14.048871 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 13375
I20260619 20:40:14.053249 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 13022
2026-06-19T20:40:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1086 ms)
[----------] 6 tests from EncryptionPolicyTest (8373 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:14 dist-test-slave-0t40 krb5kdc[13547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:14 dist-test-slave-0t40 krb5kdc[13547](info): set up 2 sockets
Jun 19 20:40:14 dist-test-slave-0t40 krb5kdc[13547](info): commencing operation
krb5kdc: starting...
W20260619 20:40:15.085304 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:15 dist-test-slave-0t40 krb5kdc[13547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:40:15.200572 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36631
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:15.279558 13560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:15.279759 13560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:15.279781 13560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:15.280918 13560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:15.280961 13560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:15.280974 13560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:15.280987 13560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:15.280998 13560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:15.282459 13560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36631
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:15.282720 13560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:15.282948 13560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:15.287745 13568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:15.287782 13566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:15.287758 13565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:15.287868 13560 server_base.cc:1034] running on GCE node
I20260619 20:40:15.288214 13560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:15.288463 13560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:15.289593 13560 hybrid_clock.cc:648] HybridClock initialized: now 1781901615289583 us; error 28 us; skew 500 ppm
Jun 19 20:40:15 dist-test-slave-0t40 krb5kdc[13547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901615, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:15.292018 13560 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:40:15.292470 13560 webserver.cc:460] Webserver started at http://127.0.0.1:33315/ using document root <none> and password file <none>
I20260619 20:40:15.292701 13560 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:15.292744 13560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:15.292874 13560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:15.293934 13560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "f4865816eb3d40d6af8d21458cd69ba6"
format_stamp: "Formatted at 2026-06-19 20:40:15 on dist-test-slave-0t40"
server_key: "3e4d0c711c4855969e55083b63e81d2b"
server_key_iv: "fabacd7ff417be2b871fa2191de2d56f"
server_key_version: "encryptionkey@0"
I20260619 20:40:15.294281 13560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "f4865816eb3d40d6af8d21458cd69ba6"
format_stamp: "Formatted at 2026-06-19 20:40:15 on dist-test-slave-0t40"
server_key: "3e4d0c711c4855969e55083b63e81d2b"
server_key_iv: "fabacd7ff417be2b871fa2191de2d56f"
server_key_version: "encryptionkey@0"
I20260619 20:40:15.295583 13560 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:40:15.296330 13575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:15.296512 13560 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:15.296600 13560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "f4865816eb3d40d6af8d21458cd69ba6"
format_stamp: "Formatted at 2026-06-19 20:40:15 on dist-test-slave-0t40"
server_key: "3e4d0c711c4855969e55083b63e81d2b"
server_key_iv: "fabacd7ff417be2b871fa2191de2d56f"
server_key_version: "encryptionkey@0"
I20260619 20:40:15.296681 13560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:15.324018 13560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:15.325981 13560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:15.326152 13560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:15.330284 13560 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36631
I20260619 20:40:15.330322 13627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36631 every 8 connection(s)
I20260619 20:40:15.330696 13560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:15.331545 13628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:15.333818 13628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6: Bootstrap starting.
I20260619 20:40:15.334517 13628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:15.334875 13628 log.cc:824] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:15.335207 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 13560
I20260619 20:40:15.335340 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:15.335566 25850 external_mini_cluster.cc:1428] Setting key 1467265b36627fbcb47f221149c23701
I20260619 20:40:15.335706 13628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6: No bootstrap required, opened a new log
I20260619 20:40:15.338083 13628 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4865816eb3d40d6af8d21458cd69ba6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36631 } }
I20260619 20:40:15.338213 13628 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:15.338234 13628 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4865816eb3d40d6af8d21458cd69ba6, State: Initialized, Role: FOLLOWER
I20260619 20:40:15.338362 13628 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4865816eb3d40d6af8d21458cd69ba6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36631 } }
I20260619 20:40:15.338442 13628 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:15.338485 13628 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:15.338532 13628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:15.339074 13628 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4865816eb3d40d6af8d21458cd69ba6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36631 } }
I20260619 20:40:15.339155 13628 leader_election.cc:302] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4865816eb3d40d6af8d21458cd69ba6; no voters:
I20260619 20:40:15.339327 13628 leader_election.cc:288] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:15.339349 13633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:15.339462 13633 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6 [term 1 LEADER]: Becoming Leader. State: Replica: f4865816eb3d40d6af8d21458cd69ba6, State: Running, Role: LEADER
I20260619 20:40:15.339572 13628 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:15.339603 13633 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4865816eb3d40d6af8d21458cd69ba6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36631 } }
I20260619 20:40:15.340062 13633 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4865816eb3d40d6af8d21458cd69ba6. Latest consensus state: current_term: 1 leader_uuid: "f4865816eb3d40d6af8d21458cd69ba6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4865816eb3d40d6af8d21458cd69ba6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36631 } } }
I20260619 20:40:15.340191 13633 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:15.340163 13635 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4865816eb3d40d6af8d21458cd69ba6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4865816eb3d40d6af8d21458cd69ba6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36631 } } }
I20260619 20:40:15.340258 13635 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:15.340440 13640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:15.340940 13640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:15.343827 13640 catalog_manager.cc:1347] Generated new cluster ID: 7e9dc27eb6544c86acca8a47434b38a0
I20260619 20:40:15.343892 13640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:40:15 dist-test-slave-0t40 krb5kdc[13547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 20:40:15.348798 13632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:15.336105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46315 (local address 127.0.0.1:36631)
0619 20:40:15.336278 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:15.336285 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:15.336309 (+ 24us) server_negotiation.cc:407] Connection header received
0619 20:40:15.336968 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:15.336975 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:15.337022 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:15.337188 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:15.337960 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:15.339357 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:15.340376 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:15.341260 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:15.341825 (+ 565us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:15.344551 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:15.344555 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:15.344555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:15.344568 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:15.346805 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:15.347336 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:15.347339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:15.347340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:15.347459 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:15.347736 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:15.347737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:15.347737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:15.348060 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:15.348211 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:15.348465 (+ 254us) server_negotiation.cc:299] Negotiation successful
0619 20:40:15.348574 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260619 20:40:15.350193 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:15.356983 13640 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:15.357537 13640 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:15.365298 13640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4865816eb3d40d6af8d21458cd69ba6: Generated new TSK 0
I20260619 20:40:15.365501 13640 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:15.371511 13632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:15.353408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44049 (local address 127.0.0.1:36631)
0619 20:40:15.353452 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:15.353454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:15.356612 (+ 3158us) server_negotiation.cc:407] Connection header received
0619 20:40:15.356701 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:15.356702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:15.356724 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:15.356808 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:15.365499 (+ 8691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:15.366054 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:15.366771 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:15.367043 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:15.367119 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:15.368636 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:15.368637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:15.368637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:15.368644 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:15.370206 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:15.370621 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:15.370622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:15.370622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:15.370680 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:15.370970 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:15.370971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:15.370972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:15.371092 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:15.371184 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:15.371367 (+ 183us) server_negotiation.cc:299] Negotiation successful
0619 20:40:15.371420 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:40:15.373762 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 13560
W20260619 20:40:15.379829 13545 connection.cc:535] client connection to 127.0.0.1:36631 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36631): Connection reset by peer (error 104)
2026-06-19T20:40:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1320 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:15 dist-test-slave-0t40 krb5kdc[13659](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:15 dist-test-slave-0t40 krb5kdc[13659](info): set up 2 sockets
Jun 19 20:40:15 dist-test-slave-0t40 krb5kdc[13659](info): commencing operation
krb5kdc: starting...
W20260619 20:40:16.409344 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:16 dist-test-slave-0t40 krb5kdc[13659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:40:16.525802 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37119
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:16.622285 13672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:16.622500 13672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:16.622525 13672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:16.623665 13672 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:16.623721 13672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:16.623739 13672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:16.623754 13672 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:16.623768 13672 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:16.625366 13672 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37119
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:16.625619 13672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:16.625872 13672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:16.630734 13678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:16.630808 13677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:16.630929 13680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:16.631311 13672 server_base.cc:1034] running on GCE node
I20260619 20:40:16.631532 13672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:16.631798 13672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:16.632952 13672 hybrid_clock.cc:648] HybridClock initialized: now 1781901616632926 us; error 36 us; skew 500 ppm
Jun 19 20:40:16 dist-test-slave-0t40 krb5kdc[13659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901616, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:16.635277 13672 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:40:16.635639 13672 webserver.cc:460] Webserver started at http://127.0.0.1:37881/ using document root <none> and password file <none>
I20260619 20:40:16.635813 13672 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:16.635852 13672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:16.635936 13672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:16.636893 13672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "155e9de72d904f5dba919b2dbce59399"
format_stamp: "Formatted at 2026-06-19 20:40:16 on dist-test-slave-0t40"
server_key: "0665c9e1d8138c048d77eaa058114932"
server_key_iv: "36317b799e1f82fded1cb206cd50bd67"
server_key_version: "encryptionkey@0"
I20260619 20:40:16.637255 13672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "155e9de72d904f5dba919b2dbce59399"
format_stamp: "Formatted at 2026-06-19 20:40:16 on dist-test-slave-0t40"
server_key: "0665c9e1d8138c048d77eaa058114932"
server_key_iv: "36317b799e1f82fded1cb206cd50bd67"
server_key_version: "encryptionkey@0"
I20260619 20:40:16.638461 13672 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:40:16.639132 13687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:16.639348 13672 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:40:16.639439 13672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "155e9de72d904f5dba919b2dbce59399"
format_stamp: "Formatted at 2026-06-19 20:40:16 on dist-test-slave-0t40"
server_key: "0665c9e1d8138c048d77eaa058114932"
server_key_iv: "36317b799e1f82fded1cb206cd50bd67"
server_key_version: "encryptionkey@0"
I20260619 20:40:16.639493 13672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:16.685168 13672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:16.687484 13672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:16.687595 13672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:16.691521 13672 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37119
I20260619 20:40:16.691623 13739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37119 every 8 connection(s)
I20260619 20:40:16.691931 13672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:16.692837 13740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:16.695127 13740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399: Bootstrap starting.
I20260619 20:40:16.695701 13740 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:16.695993 13740 log.cc:824] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:16.696677 13740 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399: No bootstrap required, opened a new log
I20260619 20:40:16.698310 13740 raft_consensus.cc:348] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "155e9de72d904f5dba919b2dbce59399" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37119 } }
I20260619 20:40:16.698469 13740 raft_consensus.cc:374] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:16.698516 13740 raft_consensus.cc:729] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 155e9de72d904f5dba919b2dbce59399, State: Initialized, Role: FOLLOWER
I20260619 20:40:16.698661 13740 consensus_queue.cc:260] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "155e9de72d904f5dba919b2dbce59399" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37119 } }
I20260619 20:40:16.698743 13740 raft_consensus.cc:388] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:16.698792 13740 raft_consensus.cc:482] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:16.698843 13740 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:16.699363 13740 raft_consensus.cc:504] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "155e9de72d904f5dba919b2dbce59399" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37119 } }
I20260619 20:40:16.699477 13740 leader_election.cc:302] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 155e9de72d904f5dba919b2dbce59399; no voters:
I20260619 20:40:16.699648 13740 leader_election.cc:288] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:16.699694 13743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:16.699898 13740 sys_catalog.cc:564] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:16.699916 13743 raft_consensus.cc:686] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399 [term 1 LEADER]: Becoming Leader. State: Replica: 155e9de72d904f5dba919b2dbce59399, State: Running, Role: LEADER
I20260619 20:40:16.700003 13743 consensus_queue.cc:237] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "155e9de72d904f5dba919b2dbce59399" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37119 } }
I20260619 20:40:16.700420 13745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 155e9de72d904f5dba919b2dbce59399. Latest consensus state: current_term: 1 leader_uuid: "155e9de72d904f5dba919b2dbce59399" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "155e9de72d904f5dba919b2dbce59399" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37119 } } }
I20260619 20:40:16.700513 13745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:16.700548 13744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "155e9de72d904f5dba919b2dbce59399" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "155e9de72d904f5dba919b2dbce59399" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37119 } } }
I20260619 20:40:16.700609 13744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:16.700770 13752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:16.700899 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 13672
I20260619 20:40:16.701017 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:16.701256 25850 external_mini_cluster.cc:1428] Setting key 2c4fe3cbf239a62ea75dc08a723b6318
I20260619 20:40:16.701330 13752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:16.703079 13752 catalog_manager.cc:1347] Generated new cluster ID: 44d209bbecb14350b7df8a1099f31b44
I20260619 20:40:16.703140 13752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:16.707320 13762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:16.701851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55099 (local address 127.0.0.1:37119)
0619 20:40:16.702041 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:16.702051 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:16.702085 (+ 34us) server_negotiation.cc:407] Connection header received
0619 20:40:16.702200 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:16.702207 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:16.702271 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:16.702409 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:16.702749 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:16.704085 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:16.705189 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:16.705800 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:16.706180 (+ 380us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:16.706612 (+ 432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:16.706616 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:16.706616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:16.706850 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:16.706891 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:16.706907 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:16.707001 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:16.707086 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 20:40:16.707116 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260619 20:40:16.708299 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:16.716830 13762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:16.713363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45875 (local address 127.0.0.1:37119)
0619 20:40:16.713408 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:16.713410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:16.713545 (+ 135us) server_negotiation.cc:407] Connection header received
0619 20:40:16.713602 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:16.713603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:16.713614 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:16.713668 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:16.713788 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:16.714508 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:16.715536 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:16.715849 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:16.715909 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:16.716583 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:16.716584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:16.716585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:16.716589 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:16.716602 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:16.716606 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:16.716659 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:16.716744 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 20:40:16.716755 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 20:40:16.722609 13752 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:16.723115 13752 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:16.737269 13752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 155e9de72d904f5dba919b2dbce59399: Generated new TSK 0
I20260619 20:40:16.737524 13752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:16.740640 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 13672
W20260619 20:40:16.744818 13657 connection.cc:535] client connection to 127.0.0.1:37119 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37119): Connection reset by peer (error 104)
2026-06-19T20:40:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1365 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:16 dist-test-slave-0t40 krb5kdc[13771](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:16 dist-test-slave-0t40 krb5kdc[13771](info): set up 2 sockets
Jun 19 20:40:16 dist-test-slave-0t40 krb5kdc[13771](info): commencing operation
krb5kdc: starting...
W20260619 20:40:17.773499 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:17 dist-test-slave-0t40 krb5kdc[13771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:40:17.891631 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34415
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:17.972705 13784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:17.972913 13784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:17.972934 13784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:17.974155 13784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:17.974200 13784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:17.974215 13784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:17.974227 13784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:17.974277 13784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:17.975735 13784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34415
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:17.975981 13784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:17.976198 13784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:17.980942 13792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:17.980971 13789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:17.980983 13790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:17.981001 13784 server_base.cc:1034] running on GCE node
I20260619 20:40:17.981340 13784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:17.981616 13784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:17.982755 13784 hybrid_clock.cc:648] HybridClock initialized: now 1781901617982745 us; error 31 us; skew 500 ppm
Jun 19 20:40:17 dist-test-slave-0t40 krb5kdc[13771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901617, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:17.985175 13784 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:40:17.985612 13784 webserver.cc:460] Webserver started at http://127.0.0.1:40371/ using document root <none> and password file <none>
I20260619 20:40:17.985834 13784 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:17.985901 13784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:17.986053 13784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:17.987075 13784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "97423dc750fa498c8022be5076c481cf"
format_stamp: "Formatted at 2026-06-19 20:40:17 on dist-test-slave-0t40"
server_key: "3828e62c1bf6d18b5e26564071432ce1"
server_key_iv: "55fb9e8deeeb2533e14d50d2cba7d415"
server_key_version: "encryptionkey@0"
I20260619 20:40:17.987435 13784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "97423dc750fa498c8022be5076c481cf"
format_stamp: "Formatted at 2026-06-19 20:40:17 on dist-test-slave-0t40"
server_key: "3828e62c1bf6d18b5e26564071432ce1"
server_key_iv: "55fb9e8deeeb2533e14d50d2cba7d415"
server_key_version: "encryptionkey@0"
I20260619 20:40:17.988706 13784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:40:17.989477 13799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:17.989639 13784 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:17.989735 13784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "97423dc750fa498c8022be5076c481cf"
format_stamp: "Formatted at 2026-06-19 20:40:17 on dist-test-slave-0t40"
server_key: "3828e62c1bf6d18b5e26564071432ce1"
server_key_iv: "55fb9e8deeeb2533e14d50d2cba7d415"
server_key_version: "encryptionkey@0"
I20260619 20:40:17.989817 13784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:17.992985 13784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:17.994726 13784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:17.994882 13784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:17.998857 13784 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34415
I20260619 20:40:17.998885 13851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34415 every 8 connection(s)
I20260619 20:40:17.999193 13784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:17.999800 13852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:18.001978 13852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf: Bootstrap starting.
I20260619 20:40:18.002532 13852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:18.002828 13852 log.cc:824] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:18.003509 13852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf: No bootstrap required, opened a new log
I20260619 20:40:18.005115 13852 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97423dc750fa498c8022be5076c481cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34415 } }
I20260619 20:40:18.005249 13852 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:18.005271 13852 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97423dc750fa498c8022be5076c481cf, State: Initialized, Role: FOLLOWER
I20260619 20:40:18.005420 13852 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97423dc750fa498c8022be5076c481cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34415 } }
I20260619 20:40:18.005479 13852 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:18.005504 13852 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:18.005530 13852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:18.006019 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 13784
I20260619 20:40:18.006103 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:18.006129 13852 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97423dc750fa498c8022be5076c481cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34415 } }
I20260619 20:40:18.006274 13852 leader_election.cc:302] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97423dc750fa498c8022be5076c481cf; no voters:
I20260619 20:40:18.006366 25850 external_mini_cluster.cc:1428] Setting key 1202cc0631dcfba1740c7c6a5b6906cb
I20260619 20:40:18.006451 13852 leader_election.cc:288] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:18.006474 13855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:18.006587 13855 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf [term 1 LEADER]: Becoming Leader. State: Replica: 97423dc750fa498c8022be5076c481cf, State: Running, Role: LEADER
I20260619 20:40:18.006704 13852 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:18.006713 13855 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97423dc750fa498c8022be5076c481cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34415 } }
I20260619 20:40:18.007046 13855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97423dc750fa498c8022be5076c481cf. Latest consensus state: current_term: 1 leader_uuid: "97423dc750fa498c8022be5076c481cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97423dc750fa498c8022be5076c481cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34415 } } }
I20260619 20:40:18.007061 13856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97423dc750fa498c8022be5076c481cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97423dc750fa498c8022be5076c481cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34415 } } }
I20260619 20:40:18.007251 13855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:18.007264 13856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:18.007513 13865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:18.008414 13865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:18.009877 13866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:18.007334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32933 (local address 127.0.0.1:34415)
0619 20:40:18.008285 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:18.008294 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:18.008335 (+ 41us) server_negotiation.cc:407] Connection header received
0619 20:40:18.008446 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:18.008453 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:18.008484 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:18.008970 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:18.009459 (+ 489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:18.009461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:18.009462 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:18.009465 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:18.009501 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:18.009514 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:18.009575 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:18.009662 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 20:40:18.009701 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":898,"thread_start_us":87,"threads_started":1}
I20260619 20:40:18.010318 13865 catalog_manager.cc:1347] Generated new cluster ID: cf964ad84f124d84b3c82fda631faab8
I20260619 20:40:18.010370 13865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:18.010811 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:18.013157 13866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:18.012167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38117 (local address 127.0.0.1:34415)
0619 20:40:18.012200 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:18.012202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:18.012327 (+ 125us) server_negotiation.cc:407] Connection header received
0619 20:40:18.012370 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:18.012371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:18.012387 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:18.012444 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:18.012932 (+ 488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:18.012933 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:18.012933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:18.012937 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:18.012950 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:18.012954 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:18.013010 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:18.013090 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 20:40:18.013101 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:40:18.027766 13865 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:18.027940 13865 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:18.034757 13865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97423dc750fa498c8022be5076c481cf: Generated new TSK 0
I20260619 20:40:18.034932 13865 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:18.039031 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 13784
2026-06-19T20:40:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1298 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:18 dist-test-slave-0t40 krb5kdc[13883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:18 dist-test-slave-0t40 krb5kdc[13883](info): set up 2 sockets
Jun 19 20:40:18 dist-test-slave-0t40 krb5kdc[13883](info): commencing operation
krb5kdc: starting...
W20260619 20:40:19.073663 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:19 dist-test-slave-0t40 krb5kdc[13883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:40:19.192494 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38205
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:19.271466 13896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:19.271664 13896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:19.271687 13896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:19.272838 13896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:19.272881 13896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:19.272894 13896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:19.272908 13896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:19.272920 13896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:19.274417 13896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:19.274675 13896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:19.274901 13896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:19.279891 13902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:19.279907 13901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:19.280088 13896 server_base.cc:1034] running on GCE node
W20260619 20:40:19.279920 13904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:19.280403 13896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:19.280704 13896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:19.281862 13896 hybrid_clock.cc:648] HybridClock initialized: now 1781901619281850 us; error 30 us; skew 500 ppm
Jun 19 20:40:19 dist-test-slave-0t40 krb5kdc[13883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901619, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:19.284124 13896 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:40:19.284518 13896 webserver.cc:460] Webserver started at http://127.0.0.1:45785/ using document root <none> and password file <none>
I20260619 20:40:19.284739 13896 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:19.284812 13896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:19.284960 13896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:19.286019 13896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "8716105e502d4bea9b9fea2615a5370f"
format_stamp: "Formatted at 2026-06-19 20:40:19 on dist-test-slave-0t40"
server_key: "70483316997439e3f64d736232d43ece"
server_key_iv: "cdd52a96478232948469d62517b84e33"
server_key_version: "encryptionkey@0"
I20260619 20:40:19.286368 13896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "8716105e502d4bea9b9fea2615a5370f"
format_stamp: "Formatted at 2026-06-19 20:40:19 on dist-test-slave-0t40"
server_key: "70483316997439e3f64d736232d43ece"
server_key_iv: "cdd52a96478232948469d62517b84e33"
server_key_version: "encryptionkey@0"
I20260619 20:40:19.287652 13896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:40:19.288389 13911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:19.288553 13896 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:40:19.288647 13896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "8716105e502d4bea9b9fea2615a5370f"
format_stamp: "Formatted at 2026-06-19 20:40:19 on dist-test-slave-0t40"
server_key: "70483316997439e3f64d736232d43ece"
server_key_iv: "cdd52a96478232948469d62517b84e33"
server_key_version: "encryptionkey@0"
I20260619 20:40:19.288729 13896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:19.299835 13896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:19.301757 13896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:19.301921 13896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:19.305620 13896 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38205
I20260619 20:40:19.305670 13963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38205 every 8 connection(s)
I20260619 20:40:19.306031 13896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:19.306895 13964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:19.307076 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 13896
I20260619 20:40:19.307188 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:19.307380 25850 external_mini_cluster.cc:1428] Setting key 5a62193cb35e13c9dc67594818fe14e4
I20260619 20:40:19.309969 13964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f: Bootstrap starting.
I20260619 20:40:19.310580 13964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:19.310884 13964 log.cc:824] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:19.311573 13964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f: No bootstrap required, opened a new log
Jun 19 20:40:19 dist-test-slave-0t40 krb5kdc[13883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 20:40:19.313892 13964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8716105e502d4bea9b9fea2615a5370f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38205 } }
I20260619 20:40:19.314069 13964 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:19.314097 13964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8716105e502d4bea9b9fea2615a5370f, State: Initialized, Role: FOLLOWER
I20260619 20:40:19.314255 13964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8716105e502d4bea9b9fea2615a5370f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38205 } }
I20260619 20:40:19.314333 13964 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:19.314357 13964 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:19.314427 13964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:19.315187 13964 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8716105e502d4bea9b9fea2615a5370f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38205 } }
I20260619 20:40:19.315316 13964 leader_election.cc:302] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8716105e502d4bea9b9fea2615a5370f; no voters:
I20260619 20:40:19.315498 13964 leader_election.cc:288] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:19.315572 13969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:19.315719 13969 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f [term 1 LEADER]: Becoming Leader. State: Replica: 8716105e502d4bea9b9fea2615a5370f, State: Running, Role: LEADER
I20260619 20:40:19.315788 13964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:19.315866 13969 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8716105e502d4bea9b9fea2615a5370f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38205 } }
I20260619 20:40:19.316325 13970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8716105e502d4bea9b9fea2615a5370f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8716105e502d4bea9b9fea2615a5370f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38205 } } }
I20260619 20:40:19.316404 13970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:19.316599 13971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8716105e502d4bea9b9fea2615a5370f. Latest consensus state: current_term: 1 leader_uuid: "8716105e502d4bea9b9fea2615a5370f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8716105e502d4bea9b9fea2615a5370f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38205 } } }
I20260619 20:40:19.316918 13971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f [sys.catalog]: This master's current role is: LEADER
W20260619 20:40:19.317785 13985 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:40:19.317879 13985 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:40:19.317965 13977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:19.318177 13977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:19.318413 13967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:19.307868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56713 (local address 127.0.0.1:38205)
0619 20:40:19.308050 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:19.308059 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:19.308092 (+ 33us) server_negotiation.cc:407] Connection header received
0619 20:40:19.308504 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:19.308510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:19.308552 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:19.308689 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:19.309397 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:19.310349 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:19.311203 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:19.311607 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:19.311743 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:19.314489 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:19.314493 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:19.314493 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:19.314507 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:19.316549 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:19.317000 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:19.317003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:19.317003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:19.317128 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:19.317426 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:19.317428 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:19.317428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:19.317733 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:19.317898 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:19.318033 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 20:40:19.318177 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
I20260619 20:40:19.319423 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:19.319741 13977 catalog_manager.cc:1347] Generated new cluster ID: 6dbcb2065d0e4683a381b1b5f94ce464
I20260619 20:40:19.319793 13977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:19.327431 13967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:19.320871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55185 (local address 127.0.0.1:38205)
0619 20:40:19.320910 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:19.320912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:19.321257 (+ 345us) server_negotiation.cc:407] Connection header received
0619 20:40:19.321337 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:19.321338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:19.321349 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:19.321412 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:19.321961 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:19.322452 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:19.323173 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:19.323398 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:19.323488 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:19.324725 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:19.324726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:19.324727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:19.324734 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:19.326226 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:19.326614 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:19.326616 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:19.326616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:19.326674 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:19.326902 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:19.326902 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:19.326903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:19.327022 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:19.327128 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:19.327249 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 20:40:19.327324 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:40:19.337335 13977 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:19.338047 13977 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:19.346985 13977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8716105e502d4bea9b9fea2615a5370f: Generated new TSK 0
I20260619 20:40:19.347183 13977 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:19.358361 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 13896
2026-06-19T20:40:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1319 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:19 dist-test-slave-0t40 krb5kdc[13995](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:19 dist-test-slave-0t40 krb5kdc[13995](info): set up 2 sockets
Jun 19 20:40:19 dist-test-slave-0t40 krb5kdc[13995](info): commencing operation
krb5kdc: starting...
W20260619 20:40:20.389406 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:20 dist-test-slave-0t40 krb5kdc[13995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:40:20.508774 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38099
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:20.588482 14008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:20.588726 14008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:20.588765 14008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:20.589923 14008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:20.589993 14008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:20.590016 14008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:20.590046 14008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:20.590078 14008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:20.591526 14008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:20.591814 14008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:20.592046 14008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:20.596927 14016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:20.596968 14014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:20.597190 14008 server_base.cc:1034] running on GCE node
W20260619 20:40:20.597023 14013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:20.597483 14008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:20.597747 14008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:20.598937 14008 hybrid_clock.cc:648] HybridClock initialized: now 1781901620598923 us; error 35 us; skew 500 ppm
Jun 19 20:40:20 dist-test-slave-0t40 krb5kdc[13995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901620, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:20.601478 14008 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:40:20.601907 14008 webserver.cc:460] Webserver started at http://127.0.0.1:44181/ using document root <none> and password file <none>
I20260619 20:40:20.602149 14008 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:20.602217 14008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:20.602371 14008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:20.603454 14008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "2314542508b34fb2acb1d5beac964a2e"
format_stamp: "Formatted at 2026-06-19 20:40:20 on dist-test-slave-0t40"
server_key: "2a27e246da59cc660a8b4ae45885e9bf"
server_key_iv: "576519bbae5875aace6d83f314ea99e1"
server_key_version: "encryptionkey@0"
I20260619 20:40:20.603821 14008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "2314542508b34fb2acb1d5beac964a2e"
format_stamp: "Formatted at 2026-06-19 20:40:20 on dist-test-slave-0t40"
server_key: "2a27e246da59cc660a8b4ae45885e9bf"
server_key_iv: "576519bbae5875aace6d83f314ea99e1"
server_key_version: "encryptionkey@0"
I20260619 20:40:20.605211 14008 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:40:20.606099 14023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:20.606294 14008 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:20.606396 14008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "2314542508b34fb2acb1d5beac964a2e"
format_stamp: "Formatted at 2026-06-19 20:40:20 on dist-test-slave-0t40"
server_key: "2a27e246da59cc660a8b4ae45885e9bf"
server_key_iv: "576519bbae5875aace6d83f314ea99e1"
server_key_version: "encryptionkey@0"
I20260619 20:40:20.606484 14008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:20.626264 14008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:20.628178 14008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:20.628350 14008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:20.632406 14008 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38099
I20260619 20:40:20.632444 14075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38099 every 8 connection(s)
I20260619 20:40:20.632817 14008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:20.633533 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 14008
I20260619 20:40:20.633648 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:20.633872 25850 external_mini_cluster.cc:1428] Setting key 000dc86cf073e64c20a160ce72afc395
I20260619 20:40:20.633909 14076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:20.637446 14076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e: Bootstrap starting.
I20260619 20:40:20.638149 14076 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:20.638500 14076 log.cc:824] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:20.639394 14076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e: No bootstrap required, opened a new log
I20260619 20:40:20.639618 14079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:20.634399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35167 (local address 127.0.0.1:38099)
0619 20:40:20.634610 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:20.634617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:20.634642 (+ 25us) server_negotiation.cc:407] Connection header received
0619 20:40:20.635048 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:20.635054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:20.635099 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:20.635213 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:20.635507 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:20.636398 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:20.637581 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:20.638015 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:20.638103 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:20.638905 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:20.638909 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:20.638910 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:20.639112 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:20.639142 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:20.639154 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:20.639206 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:20.639333 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 20:40:20.639408 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":96,"threads_started":1}
I20260619 20:40:20.640506 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:20.641986 14076 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2314542508b34fb2acb1d5beac964a2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } }
I20260619 20:40:20.642174 14076 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:20.642238 14076 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2314542508b34fb2acb1d5beac964a2e, State: Initialized, Role: FOLLOWER
I20260619 20:40:20.642376 14076 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2314542508b34fb2acb1d5beac964a2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } }
I20260619 20:40:20.642469 14076 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:20.642529 14076 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:20.642581 14076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:20.643148 14076 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2314542508b34fb2acb1d5beac964a2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } }
I20260619 20:40:20.643275 14076 leader_election.cc:302] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2314542508b34fb2acb1d5beac964a2e; no voters:
I20260619 20:40:20.643452 14076 leader_election.cc:288] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:20.643496 14086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:20.643641 14086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e [term 1 LEADER]: Becoming Leader. State: Replica: 2314542508b34fb2acb1d5beac964a2e, State: Running, Role: LEADER
I20260619 20:40:20.643716 14076 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:20.643765 14086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2314542508b34fb2acb1d5beac964a2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } }
I20260619 20:40:20.644091 14086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2314542508b34fb2acb1d5beac964a2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2314542508b34fb2acb1d5beac964a2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } } }
I20260619 20:40:20.644230 14086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:20.644408 14086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2314542508b34fb2acb1d5beac964a2e. Latest consensus state: current_term: 1 leader_uuid: "2314542508b34fb2acb1d5beac964a2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2314542508b34fb2acb1d5beac964a2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38099 } } }
I20260619 20:40:20.644522 14086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:20.644922 14094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:20.645713 14079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:20.642168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53403 (local address 127.0.0.1:38099)
0619 20:40:20.642209 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:20.642211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:20.642323 (+ 112us) server_negotiation.cc:407] Connection header received
0619 20:40:20.642434 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:20.642435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:20.642444 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:20.642519 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:20.642645 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:20.643103 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:20.644149 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:20.644434 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:20.644479 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:20.645387 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:20.645388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:20.645389 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:20.645394 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:20.645406 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:20.645409 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:20.645449 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:20.645610 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 20:40:20.645631 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:40:20.645812 14094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:20.647287 14094 catalog_manager.cc:1347] Generated new cluster ID: 4ff69b480b1f4baf9e892ef35b5c1cbd
I20260619 20:40:20.647339 14094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:20.654476 14094 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:20.655242 14094 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:20.661505 14094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2314542508b34fb2acb1d5beac964a2e: Generated new TSK 0
I20260619 20:40:20.661856 14094 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:20.667855 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 14008
2026-06-19T20:40:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:20 dist-test-slave-0t40 krb5kdc[14107](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:20 dist-test-slave-0t40 krb5kdc[14107](info): set up 2 sockets
Jun 19 20:40:20 dist-test-slave-0t40 krb5kdc[14107](info): commencing operation
krb5kdc: starting...
W20260619 20:40:21.705322 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:21 dist-test-slave-0t40 krb5kdc[14107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:40:21.823328 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39773
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:21.901758 14120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:21.901955 14120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:21.901978 14120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:21.903110 14120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:21.903151 14120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:21.903165 14120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:21.903178 14120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:21.903190 14120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:21.904639 14120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:21.904870 14120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:21.905128 14120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:21.909793 14125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:21.909794 14128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:21.909848 14126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:21.910025 14120 server_base.cc:1034] running on GCE node
I20260619 20:40:21.910310 14120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:21.910611 14120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:21.911779 14120 hybrid_clock.cc:648] HybridClock initialized: now 1781901621911776 us; error 53 us; skew 500 ppm
Jun 19 20:40:21 dist-test-slave-0t40 krb5kdc[14107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901621, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:21.917953 14120 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:40:21.918422 14120 webserver.cc:460] Webserver started at http://127.0.0.1:36849/ using document root <none> and password file <none>
I20260619 20:40:21.918641 14120 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:21.918707 14120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:21.918859 14120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:21.919878 14120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "42b139f5d6c54dbd929556726f651a7e"
format_stamp: "Formatted at 2026-06-19 20:40:21 on dist-test-slave-0t40"
server_key: "5f20c82ed6f27b8f4690fbc657bd30b9"
server_key_iv: "73394211bbd574f58f497152e52e77bc"
server_key_version: "encryptionkey@0"
I20260619 20:40:21.920228 14120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "42b139f5d6c54dbd929556726f651a7e"
format_stamp: "Formatted at 2026-06-19 20:40:21 on dist-test-slave-0t40"
server_key: "5f20c82ed6f27b8f4690fbc657bd30b9"
server_key_iv: "73394211bbd574f58f497152e52e77bc"
server_key_version: "encryptionkey@0"
I20260619 20:40:21.921775 14120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:40:21.922544 14135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:21.922730 14120 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:40:21.922807 14120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "42b139f5d6c54dbd929556726f651a7e"
format_stamp: "Formatted at 2026-06-19 20:40:21 on dist-test-slave-0t40"
server_key: "5f20c82ed6f27b8f4690fbc657bd30b9"
server_key_iv: "73394211bbd574f58f497152e52e77bc"
server_key_version: "encryptionkey@0"
I20260619 20:40:21.922890 14120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:21.926126 14120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:21.928143 14120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:21.928284 14120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:21.932310 14120 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39773
I20260619 20:40:21.932346 14187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39773 every 8 connection(s)
I20260619 20:40:21.932686 14120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:21.933751 14188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:21.935891 14188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e: Bootstrap starting.
I20260619 20:40:21.936388 14188 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:21.936705 14188 log.cc:824] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:21.937448 14188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e: No bootstrap required, opened a new log
I20260619 20:40:21.937700 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 14120
I20260619 20:40:21.937808 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:21.938002 25850 external_mini_cluster.cc:1428] Setting key 750ae204fcd851a56cbad1ec7d971a93
I20260619 20:40:21.939503 14188 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b139f5d6c54dbd929556726f651a7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39773 } }
I20260619 20:40:21.939669 14188 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:21.939708 14188 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42b139f5d6c54dbd929556726f651a7e, State: Initialized, Role: FOLLOWER
I20260619 20:40:21.939898 14188 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b139f5d6c54dbd929556726f651a7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39773 } }
I20260619 20:40:21.939989 14188 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:21.940011 14188 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:21.940074 14188 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:21.940774 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:21.938492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50955 (local address 127.0.0.1:39773)
0619 20:40:21.938659 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:21.938667 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:21.938692 (+ 25us) server_negotiation.cc:407] Connection header received
0619 20:40:21.939489 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:21.939495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:21.939520 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:21.939631 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:21.940311 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:21.940314 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:21.940314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:21.940318 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:21.940357 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:21.940368 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:21.940423 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:21.940519 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 20:40:21.940549 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260619 20:40:21.941201 14188 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b139f5d6c54dbd929556726f651a7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39773 } }
I20260619 20:40:21.941303 14188 leader_election.cc:302] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42b139f5d6c54dbd929556726f651a7e; no voters:
I20260619 20:40:21.941455 14188 leader_election.cc:288] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:21.941475 14193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:21.941624 14193 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e [term 1 LEADER]: Becoming Leader. State: Replica: 42b139f5d6c54dbd929556726f651a7e, State: Running, Role: LEADER
I20260619 20:40:21.941669 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:21.941740 14188 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:21.941761 14193 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b139f5d6c54dbd929556726f651a7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39773 } }
I20260619 20:40:21.942095 14194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42b139f5d6c54dbd929556726f651a7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b139f5d6c54dbd929556726f651a7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39773 } } }
I20260619 20:40:21.942183 14194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:21.942345 14195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42b139f5d6c54dbd929556726f651a7e. Latest consensus state: current_term: 1 leader_uuid: "42b139f5d6c54dbd929556726f651a7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b139f5d6c54dbd929556726f651a7e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39773 } } }
I20260619 20:40:21.942405 14195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:21.942659 14200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:21.943171 14200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:21.944693 14200 catalog_manager.cc:1347] Generated new cluster ID: 2d4a69b03d434d7c9ce17a79c884bd7c
I20260619 20:40:21.944746 14200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:21.950641 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:21.949408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50421 (local address 127.0.0.1:39773)
0619 20:40:21.949448 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:21.949452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:21.949633 (+ 181us) server_negotiation.cc:407] Connection header received
0619 20:40:21.949708 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:21.949710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:21.949715 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:21.949780 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:21.950429 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:21.950430 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:21.950431 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:21.950435 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:21.950450 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:21.950454 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:21.950497 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:21.950560 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 20:40:21.950575 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:40:21.952718 14200 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:21.952858 14200 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:21.958981 14200 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42b139f5d6c54dbd929556726f651a7e: Generated new TSK 0
I20260619 20:40:21.959136 14200 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:21.962096 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 14120
2026-06-19T20:40:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1293 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:22 dist-test-slave-0t40 krb5kdc[14219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:22 dist-test-slave-0t40 krb5kdc[14219](info): set up 2 sockets
Jun 19 20:40:22 dist-test-slave-0t40 krb5kdc[14219](info): commencing operation
krb5kdc: starting...
W20260619 20:40:22.993443 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:23 dist-test-slave-0t40 krb5kdc[14219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:40:23.108240 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37829
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:23.187860 14232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:23.188079 14232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:23.188118 14232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:23.189337 14232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:23.189393 14232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:23.189416 14232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:23.189446 14232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:23.189477 14232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:23.190904 14232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:23.191166 14232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:23.191395 14232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:23.196056 14238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:23.196139 14237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:23.196240 14232 server_base.cc:1034] running on GCE node
W20260619 20:40:23.196089 14240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:23.196564 14232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:23.196830 14232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:23.197983 14232 hybrid_clock.cc:648] HybridClock initialized: now 1781901623197942 us; error 51 us; skew 500 ppm
Jun 19 20:40:23 dist-test-slave-0t40 krb5kdc[14219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901623, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:23.200455 14232 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:40:23.200927 14232 webserver.cc:460] Webserver started at http://127.0.0.1:45155/ using document root <none> and password file <none>
I20260619 20:40:23.201205 14232 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:23.201275 14232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:23.201431 14232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:23.202557 14232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "5c81168756fe47dca64de84c9f8a10d4"
format_stamp: "Formatted at 2026-06-19 20:40:23 on dist-test-slave-0t40"
server_key: "c9a69400561d335806b438a843373131"
server_key_iv: "11c68d95ac87b46d81eba0dda32eeb31"
server_key_version: "encryptionkey@0"
I20260619 20:40:23.202976 14232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "5c81168756fe47dca64de84c9f8a10d4"
format_stamp: "Formatted at 2026-06-19 20:40:23 on dist-test-slave-0t40"
server_key: "c9a69400561d335806b438a843373131"
server_key_iv: "11c68d95ac87b46d81eba0dda32eeb31"
server_key_version: "encryptionkey@0"
I20260619 20:40:23.204722 14232 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 20:40:23.205715 14247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:23.205883 14232 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:40:23.205962 14232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "5c81168756fe47dca64de84c9f8a10d4"
format_stamp: "Formatted at 2026-06-19 20:40:23 on dist-test-slave-0t40"
server_key: "c9a69400561d335806b438a843373131"
server_key_iv: "11c68d95ac87b46d81eba0dda32eeb31"
server_key_version: "encryptionkey@0"
I20260619 20:40:23.206017 14232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:23.227334 14232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:23.229288 14232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:23.229398 14232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:23.233654 14232 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37829
I20260619 20:40:23.233714 14299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37829 every 8 connection(s)
I20260619 20:40:23.234030 14232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:23.234944 14300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:23.237396 14300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4: Bootstrap starting.
I20260619 20:40:23.238034 14300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:23.238356 14300 log.cc:824] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:23.239140 14300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4: No bootstrap required, opened a new log
I20260619 20:40:23.240765 14300 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c81168756fe47dca64de84c9f8a10d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37829 } }
I20260619 20:40:23.240907 14300 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:23.240931 14300 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c81168756fe47dca64de84c9f8a10d4, State: Initialized, Role: FOLLOWER
I20260619 20:40:23.241103 14300 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c81168756fe47dca64de84c9f8a10d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37829 } }
I20260619 20:40:23.241168 14300 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:23.241195 14300 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:23.241248 14300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:23.241793 14300 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c81168756fe47dca64de84c9f8a10d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37829 } }
I20260619 20:40:23.241901 14300 leader_election.cc:302] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c81168756fe47dca64de84c9f8a10d4; no voters:
I20260619 20:40:23.242084 14300 leader_election.cc:288] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:23.242122 14303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:23.242326 14303 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4 [term 1 LEADER]: Becoming Leader. State: Replica: 5c81168756fe47dca64de84c9f8a10d4, State: Running, Role: LEADER
I20260619 20:40:23.242350 14300 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:23.242416 14303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c81168756fe47dca64de84c9f8a10d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37829 } }
I20260619 20:40:23.242841 14304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c81168756fe47dca64de84c9f8a10d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c81168756fe47dca64de84c9f8a10d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37829 } } }
I20260619 20:40:23.242870 14305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c81168756fe47dca64de84c9f8a10d4. Latest consensus state: current_term: 1 leader_uuid: "5c81168756fe47dca64de84c9f8a10d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c81168756fe47dca64de84c9f8a10d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37829 } } }
I20260619 20:40:23.243000 14305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:23.243137 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 14232
I20260619 20:40:23.243237 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:23.243366 14304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:23.243448 25850 external_mini_cluster.cc:1428] Setting key e38cbe2a7c3719722c9e1282691d1b1b
I20260619 20:40:23.243517 14307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:23.244029 14307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:23.245618 14307 catalog_manager.cc:1347] Generated new cluster ID: 45e5f214dbbe4632ae97fcb5ef23258c
I20260619 20:40:23.245669 14307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:40:23 dist-test-slave-0t40 krb5kdc[14219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 20:40:23.256218 14322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:23.244078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.228:39387 (local address 127.0.0.1:37829)
0619 20:40:23.244329 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:23.244339 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:23.244372 (+ 33us) server_negotiation.cc:407] Connection header received
0619 20:40:23.244700 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:23.244708 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:23.244769 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:23.245712 (+ 943us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:23.245818 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:23.247237 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:23.248352 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:23.248981 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:23.249441 (+ 460us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:23.251995 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:23.252000 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:23.252000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:23.252013 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:23.254036 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:23.254531 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:23.254534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:23.254534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:23.254651 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:23.254908 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:23.254908 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:23.254909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:23.255216 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:23.255799 (+ 583us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:23.255883 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 20:40:23.255993 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":73,"threads_started":1}
I20260619 20:40:23.257277 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:23.259133 14307 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:23.259773 14307 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:23.265792 14322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:23.258663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.228:37755 (local address 127.0.0.1:37829)
0619 20:40:23.258696 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:23.258698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:23.258800 (+ 102us) server_negotiation.cc:407] Connection header received
0619 20:40:23.258870 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:23.258871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:23.258883 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:23.258938 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:23.259560 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:23.260370 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:23.261367 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:23.261696 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:23.261768 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:23.263068 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:23.263069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:23.263070 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:23.263076 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:23.264595 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:23.264988 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:23.264989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:23.264989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:23.265072 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:23.265294 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:23.265295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:23.265295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:23.265418 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:23.265525 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:23.265642 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 20:40:23.265702 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":16}
I20260619 20:40:23.269373 14307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c81168756fe47dca64de84c9f8a10d4: Generated new TSK 0
I20260619 20:40:23.269548 14307 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:23.271991 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 14232
W20260619 20:40:23.276006 14217 connection.cc:535] client connection to 127.0.0.1:37829 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37829): Connection reset by peer (error 104)
2026-06-19T20:40:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1309 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:23 dist-test-slave-0t40 krb5kdc[14331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:23 dist-test-slave-0t40 krb5kdc[14331](info): set up 2 sockets
Jun 19 20:40:23 dist-test-slave-0t40 krb5kdc[14331](info): commencing operation
krb5kdc: starting...
W20260619 20:40:24.305552 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:24 dist-test-slave-0t40 krb5kdc[14331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:40:24.421624 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38745
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:24.502696 14344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:24.502899 14344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:24.502923 14344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:24.504065 14344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:24.504106 14344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:24.504120 14344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:24.504133 14344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:24.504145 14344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:24.505750 14344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:24.506001 14344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:24.506229 14344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:24.511091 14349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:24.511085 14352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:24.511085 14350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:24.511121 14344 server_base.cc:1034] running on GCE node
I20260619 20:40:24.511587 14344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:24.511845 14344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:24.512982 14344 hybrid_clock.cc:648] HybridClock initialized: now 1781901624512964 us; error 33 us; skew 500 ppm
Jun 19 20:40:24 dist-test-slave-0t40 krb5kdc[14331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901624, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:24.515580 14344 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:40:24.516026 14344 webserver.cc:460] Webserver started at http://127.0.0.1:37453/ using document root <none> and password file <none>
I20260619 20:40:24.516274 14344 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:24.516343 14344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:24.516487 14344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:24.517583 14344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "c09837ab1b274bf4b3a004b96f4d8644"
format_stamp: "Formatted at 2026-06-19 20:40:24 on dist-test-slave-0t40"
server_key: "026e7fff7a2a61006c5403214f0057bd"
server_key_iv: "7aa890f790d1e959eaa5a50fe2e2b146"
server_key_version: "encryptionkey@0"
I20260619 20:40:24.517952 14344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "c09837ab1b274bf4b3a004b96f4d8644"
format_stamp: "Formatted at 2026-06-19 20:40:24 on dist-test-slave-0t40"
server_key: "026e7fff7a2a61006c5403214f0057bd"
server_key_iv: "7aa890f790d1e959eaa5a50fe2e2b146"
server_key_version: "encryptionkey@0"
I20260619 20:40:24.519275 14344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:40:24.519996 14359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:24.520181 14344 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:24.520280 14344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "c09837ab1b274bf4b3a004b96f4d8644"
format_stamp: "Formatted at 2026-06-19 20:40:24 on dist-test-slave-0t40"
server_key: "026e7fff7a2a61006c5403214f0057bd"
server_key_iv: "7aa890f790d1e959eaa5a50fe2e2b146"
server_key_version: "encryptionkey@0"
I20260619 20:40:24.520359 14344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:24.541320 14344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:24.543326 14344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:24.543491 14344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:24.547425 14344 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38745
I20260619 20:40:24.547477 14411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38745 every 8 connection(s)
I20260619 20:40:24.547840 14344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:24.548653 14412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:24.550907 14412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644: Bootstrap starting.
I20260619 20:40:24.551496 14412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:24.551774 14412 log.cc:824] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:24.552464 14412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644: No bootstrap required, opened a new log
I20260619 20:40:24.554080 14412 raft_consensus.cc:348] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c09837ab1b274bf4b3a004b96f4d8644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38745 } }
I20260619 20:40:24.554214 14412 raft_consensus.cc:374] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:24.554236 14412 raft_consensus.cc:729] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c09837ab1b274bf4b3a004b96f4d8644, State: Initialized, Role: FOLLOWER
I20260619 20:40:24.554415 14412 consensus_queue.cc:260] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c09837ab1b274bf4b3a004b96f4d8644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38745 } }
I20260619 20:40:24.554507 14412 raft_consensus.cc:388] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:24.554551 14412 raft_consensus.cc:482] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:24.554600 14412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:24.555119 14412 raft_consensus.cc:504] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c09837ab1b274bf4b3a004b96f4d8644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38745 } }
I20260619 20:40:24.555231 14412 leader_election.cc:302] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c09837ab1b274bf4b3a004b96f4d8644; no voters:
I20260619 20:40:24.555414 14412 leader_election.cc:288] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:24.555454 14415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:24.555624 14415 raft_consensus.cc:686] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644 [term 1 LEADER]: Becoming Leader. State: Replica: c09837ab1b274bf4b3a004b96f4d8644, State: Running, Role: LEADER
I20260619 20:40:24.555667 14412 sys_catalog.cc:564] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:24.555815 14415 consensus_queue.cc:237] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c09837ab1b274bf4b3a004b96f4d8644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38745 } }
I20260619 20:40:24.556337 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 14344
I20260619 20:40:24.556442 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:24.556607 25850 external_mini_cluster.cc:1428] Setting key 284455d550004b2a467e290b652a7d97
I20260619 20:40:24.557183 14417 sys_catalog.cc:455] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c09837ab1b274bf4b3a004b96f4d8644. Latest consensus state: current_term: 1 leader_uuid: "c09837ab1b274bf4b3a004b96f4d8644" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c09837ab1b274bf4b3a004b96f4d8644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38745 } } }
I20260619 20:40:24.557504 14417 sys_catalog.cc:458] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:24.557224 14416 sys_catalog.cc:455] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c09837ab1b274bf4b3a004b96f4d8644" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c09837ab1b274bf4b3a004b96f4d8644" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38745 } } }
I20260619 20:40:24.557578 14416 sys_catalog.cc:458] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644 [sys.catalog]: This master's current role is: LEADER
W20260619 20:40:24.557638 14429 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:40:24.557698 14429 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:40:24.557961 14434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:24.558130 14434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:24.559697 14434 catalog_manager.cc:1347] Generated new cluster ID: 3168b4e2713c49e5b5d9caec7cd98af9
I20260619 20:40:24.559748 14434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:24.562732 14431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:24.557284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.228:38853 (local address 127.0.0.1:38745)
0619 20:40:24.557526 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:24.557536 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:24.557804 (+ 268us) server_negotiation.cc:407] Connection header received
0619 20:40:24.557958 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:24.557964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:24.558043 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:24.558210 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:24.558437 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:24.559445 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:24.560534 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:24.561221 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:24.561337 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:24.562061 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:24.562066 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:24.562066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:24.562270 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:24.562308 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:24.562323 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:24.562415 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:24.562500 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 20:40:24.562526 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":83,"threads_started":1}
I20260619 20:40:24.563694 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:24.568905 14434 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:24.569636 14434 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:24.572449 14431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:24.568847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.228:46975 (local address 127.0.0.1:38745)
0619 20:40:24.568884 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:24.568886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:24.569125 (+ 239us) server_negotiation.cc:407] Connection header received
0619 20:40:24.569184 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:24.569185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:24.569195 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:24.569258 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:24.569422 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:24.570105 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:24.571145 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:24.571444 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:24.571501 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:24.572204 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:24.572205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:24.572206 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:24.572210 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:24.572224 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:24.572228 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:24.572285 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:24.572362 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 20:40:24.572374 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:40:24.575429 14434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c09837ab1b274bf4b3a004b96f4d8644: Generated new TSK 0
I20260619 20:40:24.575604 14434 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:24.580026 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 14344
W20260619 20:40:24.584219 14329 connection.cc:535] client connection to 127.0.0.1:38745 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38745): Connection reset by peer (error 104)
2026-06-19T20:40:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1308 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:24 dist-test-slave-0t40 krb5kdc[14443](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:24 dist-test-slave-0t40 krb5kdc[14443](info): set up 2 sockets
Jun 19 20:40:24 dist-test-slave-0t40 krb5kdc[14443](info): commencing operation
krb5kdc: starting...
W20260619 20:40:25.617488 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:25 dist-test-slave-0t40 krb5kdc[14443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:40:25.729624 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44133
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:25.808862 14456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:25.809078 14456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:25.809101 14456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:25.810237 14456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:25.810279 14456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:25.810294 14456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:25.810307 14456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:25.810357 14456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:25.811766 14456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:25.812008 14456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:25.812233 14456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:25.817020 14464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:25.817034 14462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:25.817081 14456 server_base.cc:1034] running on GCE node
W20260619 20:40:25.817059 14461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:25.817481 14456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:25.817761 14456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:25.818924 14456 hybrid_clock.cc:648] HybridClock initialized: now 1781901625818916 us; error 35 us; skew 500 ppm
Jun 19 20:40:25 dist-test-slave-0t40 krb5kdc[14443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901625, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:25.821489 14456 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:40:25.821874 14456 webserver.cc:460] Webserver started at http://127.0.0.1:37209/ using document root <none> and password file <none>
I20260619 20:40:25.822052 14456 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:25.822095 14456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:25.822186 14456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:25.823179 14456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "59398fe4e22d4819ac0fa4eb652a8a5e"
format_stamp: "Formatted at 2026-06-19 20:40:25 on dist-test-slave-0t40"
server_key: "21e41de181ba561d57d7ec8f3aeac359"
server_key_iv: "9b11730aba80bc5c2eff4ac0d087cf7f"
server_key_version: "encryptionkey@0"
I20260619 20:40:25.823500 14456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "59398fe4e22d4819ac0fa4eb652a8a5e"
format_stamp: "Formatted at 2026-06-19 20:40:25 on dist-test-slave-0t40"
server_key: "21e41de181ba561d57d7ec8f3aeac359"
server_key_iv: "9b11730aba80bc5c2eff4ac0d087cf7f"
server_key_version: "encryptionkey@0"
I20260619 20:40:25.824918 14456 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:40:25.825726 14471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:25.825929 14456 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:25.826043 14456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "59398fe4e22d4819ac0fa4eb652a8a5e"
format_stamp: "Formatted at 2026-06-19 20:40:25 on dist-test-slave-0t40"
server_key: "21e41de181ba561d57d7ec8f3aeac359"
server_key_iv: "9b11730aba80bc5c2eff4ac0d087cf7f"
server_key_version: "encryptionkey@0"
I20260619 20:40:25.826120 14456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:25.829348 14456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:25.831117 14456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:25.831271 14456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:25.835083 14456 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44133
I20260619 20:40:25.835122 14523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44133 every 8 connection(s)
I20260619 20:40:25.835430 14456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:25.836381 14524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:25.838562 14524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e: Bootstrap starting.
I20260619 20:40:25.839121 14524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:25.839396 14524 log.cc:824] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:25.840116 14524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e: No bootstrap required, opened a new log
I20260619 20:40:25.841854 14524 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59398fe4e22d4819ac0fa4eb652a8a5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44133 } }
I20260619 20:40:25.841982 14524 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:25.842003 14524 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59398fe4e22d4819ac0fa4eb652a8a5e, State: Initialized, Role: FOLLOWER
I20260619 20:40:25.842168 14524 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59398fe4e22d4819ac0fa4eb652a8a5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44133 } }
I20260619 20:40:25.842247 14524 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:25.842268 14524 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:25.842327 14524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:25.842850 14524 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59398fe4e22d4819ac0fa4eb652a8a5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44133 } }
I20260619 20:40:25.842959 14524 leader_election.cc:302] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59398fe4e22d4819ac0fa4eb652a8a5e; no voters:
I20260619 20:40:25.843139 14524 leader_election.cc:288] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:25.843210 14527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:25.843374 14524 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:25.843382 14527 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e [term 1 LEADER]: Becoming Leader. State: Replica: 59398fe4e22d4819ac0fa4eb652a8a5e, State: Running, Role: LEADER
I20260619 20:40:25.843494 14527 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59398fe4e22d4819ac0fa4eb652a8a5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44133 } }
I20260619 20:40:25.843986 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 14456
I20260619 20:40:25.844110 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:25.844137 14528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59398fe4e22d4819ac0fa4eb652a8a5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59398fe4e22d4819ac0fa4eb652a8a5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44133 } } }
I20260619 20:40:25.844206 14528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:25.844350 25850 external_mini_cluster.cc:1428] Setting key 0bce37cbab907c377dfdc6a510c0e973
I20260619 20:40:25.844537 14529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59398fe4e22d4819ac0fa4eb652a8a5e. Latest consensus state: current_term: 1 leader_uuid: "59398fe4e22d4819ac0fa4eb652a8a5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59398fe4e22d4819ac0fa4eb652a8a5e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44133 } } }
I20260619 20:40:25.844611 14529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e [sys.catalog]: This master's current role is: LEADER
W20260619 20:40:25.845441 14542 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:40:25.845541 14542 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:40:25.845934 14536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:25.846107 14536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:25.846498 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:25.844779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.228:51891 (local address 127.0.0.1:44133)
0619 20:40:25.845054 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:25.845065 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:25.845101 (+ 36us) server_negotiation.cc:407] Connection header received
0619 20:40:25.845211 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:25.845217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:25.845246 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:25.845386 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:25.846045 (+ 659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:25.846048 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:25.846049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:25.846052 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:25.846099 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:25.846110 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:25.846188 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:25.846272 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 20:40:25.846317 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":64,"threads_started":1}
I20260619 20:40:25.847420 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:25.848695 14536 catalog_manager.cc:1347] Generated new cluster ID: b51cfabde2f5413b82acd602e3bf66c5
I20260619 20:40:25.848752 14536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:25.854233 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:25.849294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.228:55413 (local address 127.0.0.1:44133)
0619 20:40:25.849329 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:25.849331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:25.853243 (+ 3912us) server_negotiation.cc:407] Connection header received
0619 20:40:25.853325 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:25.853326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:25.853331 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:25.853376 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:25.854011 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:25.854012 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:25.854013 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:25.854017 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:25.854030 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:25.854035 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:25.854080 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:25.854133 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 20:40:25.854146 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:40:25.857709 14536 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:25.857853 14536 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:25.863399 14536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59398fe4e22d4819ac0fa4eb652a8a5e: Generated new TSK 0
I20260619 20:40:25.863559 14536 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:25.870119 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 14456
2026-06-19T20:40:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1290 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:25 dist-test-slave-0t40 krb5kdc[14555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:25 dist-test-slave-0t40 krb5kdc[14555](info): set up 2 sockets
Jun 19 20:40:25 dist-test-slave-0t40 krb5kdc[14555](info): commencing operation
krb5kdc: starting...
W20260619 20:40:26.921633 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:27 dist-test-slave-0t40 krb5kdc[14555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:40:27.036267 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46741
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:27.113837 14568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:27.114071 14568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:27.114109 14568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:27.115253 14568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:27.115324 14568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:27.115347 14568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:27.115374 14568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:27.115406 14568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:27.116827 14568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46741
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:27.117120 14568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:27.117339 14568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:27.122026 14576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:27.122077 14574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:27.122026 14573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:27.122459 14568 server_base.cc:1034] running on GCE node
I20260619 20:40:27.122737 14568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:27.123049 14568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:27.124213 14568 hybrid_clock.cc:648] HybridClock initialized: now 1781901627124200 us; error 28 us; skew 500 ppm
Jun 19 20:40:27 dist-test-slave-0t40 krb5kdc[14555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901627, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:27.131228 14568 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:40:27.131623 14568 webserver.cc:460] Webserver started at http://127.0.0.1:38533/ using document root <none> and password file <none>
I20260619 20:40:27.131825 14568 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:27.131868 14568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:27.131955 14568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:27.132965 14568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "44c8655781f74085923831ad4223c7fb"
format_stamp: "Formatted at 2026-06-19 20:40:27 on dist-test-slave-0t40"
server_key: "36592479b309df78efd5aeaa8ee85eca"
server_key_iv: "5e653ea6639aa5f8143d5a20858a1bc7"
server_key_version: "encryptionkey@0"
I20260619 20:40:27.133354 14568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "44c8655781f74085923831ad4223c7fb"
format_stamp: "Formatted at 2026-06-19 20:40:27 on dist-test-slave-0t40"
server_key: "36592479b309df78efd5aeaa8ee85eca"
server_key_iv: "5e653ea6639aa5f8143d5a20858a1bc7"
server_key_version: "encryptionkey@0"
I20260619 20:40:27.134640 14568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:27.135339 14583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:27.135493 14568 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:40:27.135578 14568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "44c8655781f74085923831ad4223c7fb"
format_stamp: "Formatted at 2026-06-19 20:40:27 on dist-test-slave-0t40"
server_key: "36592479b309df78efd5aeaa8ee85eca"
server_key_iv: "5e653ea6639aa5f8143d5a20858a1bc7"
server_key_version: "encryptionkey@0"
I20260619 20:40:27.135624 14568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:27.159485 14568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:27.161214 14568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:27.161317 14568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:27.164913 14568 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46741
I20260619 20:40:27.164983 14635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46741 every 8 connection(s)
I20260619 20:40:27.165294 14568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:27.166122 14636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:27.168251 14636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb: Bootstrap starting.
I20260619 20:40:27.168800 14636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:27.169137 14636 log.cc:824] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:27.169795 14636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb: No bootstrap required, opened a new log
I20260619 20:40:27.170717 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 14568
I20260619 20:40:27.170836 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:27.171037 25850 external_mini_cluster.cc:1428] Setting key 1c730e539923f552c5ff8480a4c274e0
I20260619 20:40:27.171838 14636 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c8655781f74085923831ad4223c7fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46741 } }
I20260619 20:40:27.171998 14636 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:27.172039 14636 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44c8655781f74085923831ad4223c7fb, State: Initialized, Role: FOLLOWER
I20260619 20:40:27.172183 14636 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c8655781f74085923831ad4223c7fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46741 } }
I20260619 20:40:27.172269 14636 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:27.172291 14636 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:27.172360 14636 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:27.173401 14636 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c8655781f74085923831ad4223c7fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46741 } }
I20260619 20:40:27.173501 14636 leader_election.cc:302] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44c8655781f74085923831ad4223c7fb; no voters:
I20260619 20:40:27.173635 14636 leader_election.cc:288] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:27.173696 14641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:27.173852 14636 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:27.173859 14641 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb [term 1 LEADER]: Becoming Leader. State: Replica: 44c8655781f74085923831ad4223c7fb, State: Running, Role: LEADER
I20260619 20:40:27.174017 14641 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c8655781f74085923831ad4223c7fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46741 } }
I20260619 20:40:27.174369 14642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44c8655781f74085923831ad4223c7fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c8655781f74085923831ad4223c7fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46741 } } }
I20260619 20:40:27.174386 14643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44c8655781f74085923831ad4223c7fb. Latest consensus state: current_term: 1 leader_uuid: "44c8655781f74085923831ad4223c7fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c8655781f74085923831ad4223c7fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46741 } } }
I20260619 20:40:27.174553 14642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:27.174566 14643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:27.174816 14645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:27.175411 14645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 20:40:27 dist-test-slave-0t40 krb5kdc[14555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 20:40:27.178153 14645 catalog_manager.cc:1347] Generated new cluster ID: 898418b550fc4d56868bce957c602e93
I20260619 20:40:27.178203 14645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:27.182235 14640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:27.171518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.228:59261 (local address 127.0.0.1:46741)
0619 20:40:27.171702 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:27.171712 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:27.171759 (+ 47us) server_negotiation.cc:407] Connection header received
0619 20:40:27.171892 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:27.171899 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:27.171953 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:27.172089 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:27.172806 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:27.174221 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:27.175287 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:27.175872 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:27.176003 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:27.178578 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:27.178581 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:27.178582 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:27.178592 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:27.180509 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:27.180935 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:27.180938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:27.180938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:27.181092 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:27.181327 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:27.181328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:27.181329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:27.181651 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:27.181819 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:27.181903 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 20:40:27.182009 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":70,"threads_started":1}
I20260619 20:40:27.183228 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:27.192184 14640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:27.184598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.228:34745 (local address 127.0.0.1:46741)
0619 20:40:27.184637 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:27.184639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:27.184788 (+ 149us) server_negotiation.cc:407] Connection header received
0619 20:40:27.184816 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:27.184817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:27.184826 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:27.184890 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:27.185492 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:27.186222 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:27.187180 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:27.187546 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:27.187611 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:27.188874 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:27.188875 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:27.188876 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:27.188882 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:27.190991 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:27.191407 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:27.191408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:27.191408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:27.191465 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:27.191698 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:27.191699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:27.191699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:27.191819 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:27.191918 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:27.192032 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 20:40:27.192093 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:40:27.193017 14645 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:27.193732 14645 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:27.202775 14645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44c8655781f74085923831ad4223c7fb: Generated new TSK 0
I20260619 20:40:27.202947 14645 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:27.207813 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 14568
W20260619 20:40:27.211699 14553 connection.cc:535] client connection to 127.0.0.1:46741 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46741): Connection reset by peer (error 104)
2026-06-19T20:40:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1337 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:27 dist-test-slave-0t40 krb5kdc[14667](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:27 dist-test-slave-0t40 krb5kdc[14667](info): set up 2 sockets
Jun 19 20:40:27 dist-test-slave-0t40 krb5kdc[14667](info): commencing operation
krb5kdc: starting...
W20260619 20:40:28.257602 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:28 dist-test-slave-0t40 krb5kdc[14667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:40:28.374186 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37229
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:28.454993 14680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:28.455199 14680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:28.455221 14680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:28.456329 14680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:28.456363 14680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:28.456377 14680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:28.456390 14680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:28.456403 14680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:28.457880 14680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37229
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:28.458118 14680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:28.458349 14680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:28.463102 14688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:28.463083 14686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:28.463173 14680 server_base.cc:1034] running on GCE node
W20260619 20:40:28.463083 14685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:28.463547 14680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:28.463905 14680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:28.465091 14680 hybrid_clock.cc:648] HybridClock initialized: now 1781901628465067 us; error 37 us; skew 500 ppm
Jun 19 20:40:28 dist-test-slave-0t40 krb5kdc[14667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901628, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:28.467607 14680 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:40:28.468063 14680 webserver.cc:460] Webserver started at http://127.0.0.1:38961/ using document root <none> and password file <none>
I20260619 20:40:28.468315 14680 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:28.468387 14680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:28.468538 14680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:28.469655 14680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "5c8e6cbde9574ace956ece5e351f58d8"
format_stamp: "Formatted at 2026-06-19 20:40:28 on dist-test-slave-0t40"
server_key: "ec8d7a7f502089b4281910ece0a03c7d"
server_key_iv: "303c2ee00ce81076f30c3257c1ac765d"
server_key_version: "encryptionkey@0"
I20260619 20:40:28.470036 14680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "5c8e6cbde9574ace956ece5e351f58d8"
format_stamp: "Formatted at 2026-06-19 20:40:28 on dist-test-slave-0t40"
server_key: "ec8d7a7f502089b4281910ece0a03c7d"
server_key_iv: "303c2ee00ce81076f30c3257c1ac765d"
server_key_version: "encryptionkey@0"
I20260619 20:40:28.471405 14680 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:40:28.472186 14695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:28.472359 14680 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:40:28.472465 14680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "5c8e6cbde9574ace956ece5e351f58d8"
format_stamp: "Formatted at 2026-06-19 20:40:28 on dist-test-slave-0t40"
server_key: "ec8d7a7f502089b4281910ece0a03c7d"
server_key_iv: "303c2ee00ce81076f30c3257c1ac765d"
server_key_version: "encryptionkey@0"
I20260619 20:40:28.472550 14680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:28.483997 14680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:28.486222 14680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:28.486388 14680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:28.490321 14680 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37229
I20260619 20:40:28.490370 14747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37229 every 8 connection(s)
I20260619 20:40:28.490741 14680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:28.491663 14748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:28.494022 14748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8: Bootstrap starting.
I20260619 20:40:28.494612 14748 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:28.494910 14748 log.cc:824] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:28.495592 14748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8: No bootstrap required, opened a new log
I20260619 20:40:28.497242 14748 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c8e6cbde9574ace956ece5e351f58d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37229 } }
I20260619 20:40:28.497422 14748 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:28.497500 14748 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c8e6cbde9574ace956ece5e351f58d8, State: Initialized, Role: FOLLOWER
I20260619 20:40:28.497673 14748 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c8e6cbde9574ace956ece5e351f58d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37229 } }
I20260619 20:40:28.497758 14748 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:28.497800 14748 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:28.497856 14748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:28.498392 14748 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c8e6cbde9574ace956ece5e351f58d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37229 } }
I20260619 20:40:28.498502 14748 leader_election.cc:302] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c8e6cbde9574ace956ece5e351f58d8; no voters:
I20260619 20:40:28.498672 14748 leader_election.cc:288] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:28.498731 14751 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:28.498756 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 14680
I20260619 20:40:28.498857 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:28.498888 14751 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8 [term 1 LEADER]: Becoming Leader. State: Replica: 5c8e6cbde9574ace956ece5e351f58d8, State: Running, Role: LEADER
I20260619 20:40:28.498952 14748 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:28.499053 25850 external_mini_cluster.cc:1428] Setting key c6a750557a0aa39e02333ac6ca8a1657
I20260619 20:40:28.498965 14751 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c8e6cbde9574ace956ece5e351f58d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37229 } }
I20260619 20:40:28.499389 14751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c8e6cbde9574ace956ece5e351f58d8. Latest consensus state: current_term: 1 leader_uuid: "5c8e6cbde9574ace956ece5e351f58d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c8e6cbde9574ace956ece5e351f58d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37229 } } }
I20260619 20:40:28.499471 14751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:28.499696 14753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c8e6cbde9574ace956ece5e351f58d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c8e6cbde9574ace956ece5e351f58d8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37229 } } }
I20260619 20:40:28.499790 14753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:28.499856 14758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:28.500391 14758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:28.501967 14758 catalog_manager.cc:1347] Generated new cluster ID: 9298c026d40142d8bd681abadb6a2013
I20260619 20:40:28.502018 14758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:28.505875 14762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:28.499732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.228:50483 (local address 127.0.0.1:37229)
0619 20:40:28.500308 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:28.500317 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:28.500368 (+ 51us) server_negotiation.cc:407] Connection header received
0619 20:40:28.500499 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:28.500506 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:28.500570 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:28.500981 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:28.501100 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:28.502479 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:28.503554 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:28.504199 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:28.504321 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:28.505178 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:28.505182 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:28.505183 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:28.505394 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:28.505434 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:28.505449 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:28.505545 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:28.505630 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 20:40:28.505658 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":75,"threads_started":1}
I20260619 20:40:28.506863 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:28.511741 14762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:28.508253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.228:41813 (local address 127.0.0.1:37229)
0619 20:40:28.508287 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:28.508289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:28.508387 (+ 98us) server_negotiation.cc:407] Connection header received
0619 20:40:28.508431 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:28.508432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:28.508442 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:28.508511 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:28.508622 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:28.509404 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:28.510377 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:28.510698 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:28.510770 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:28.511489 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:28.511491 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:28.511491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:28.511496 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:28.511512 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:28.511516 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:28.511580 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:28.511649 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 20:40:28.511661 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:40:28.523550 14758 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:28.524269 14758 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:28.529963 14758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c8e6cbde9574ace956ece5e351f58d8: Generated new TSK 0
I20260619 20:40:28.530156 14758 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:28.539501 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 14680
W20260619 20:40:28.543705 14665 connection.cc:535] client connection to 127.0.0.1:37229 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37229): Connection reset by peer (error 104)
2026-06-19T20:40:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1332 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:28 dist-test-slave-0t40 krb5kdc[14779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:28 dist-test-slave-0t40 krb5kdc[14779](info): set up 2 sockets
Jun 19 20:40:28 dist-test-slave-0t40 krb5kdc[14779](info): commencing operation
krb5kdc: starting...
W20260619 20:40:29.581533 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:29 dist-test-slave-0t40 krb5kdc[14779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 20:40:29.693842 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38319
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:29.772643 14792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:29.772836 14792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:29.772859 14792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:29.773931 14792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:29.773967 14792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:29.773983 14792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:29.773995 14792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:29.774008 14792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:29.775375 14792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38319
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:29.775550 14792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:29.775707 14792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:40:29.780412 14792 server_base.cc:1034] running on GCE node
W20260619 20:40:29.780366 14798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:29.780357 14797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:29.780366 14800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:29.780844 14792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:29.781177 14792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:29.782337 14792 hybrid_clock.cc:648] HybridClock initialized: now 1781901629782316 us; error 34 us; skew 500 ppm
Jun 19 20:40:29 dist-test-slave-0t40 krb5kdc[14779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901629, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:29.784718 14792 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 20:40:29.785140 14792 webserver.cc:460] Webserver started at http://127.0.0.1:40531/ using document root <none> and password file <none>
I20260619 20:40:29.785315 14792 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:29.785354 14792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:29.785437 14792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:29.786347 14792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "c99b8fb9c301449982c840607b844062"
format_stamp: "Formatted at 2026-06-19 20:40:29 on dist-test-slave-0t40"
server_key: "4adc9b13eba5889887c121d48df08de9"
server_key_iv: "72191a7c775d113966fb0d54dfe76012"
server_key_version: "encryptionkey@0"
I20260619 20:40:29.786633 14792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "c99b8fb9c301449982c840607b844062"
format_stamp: "Formatted at 2026-06-19 20:40:29 on dist-test-slave-0t40"
server_key: "4adc9b13eba5889887c121d48df08de9"
server_key_iv: "72191a7c775d113966fb0d54dfe76012"
server_key_version: "encryptionkey@0"
I20260619 20:40:29.787846 14792 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:40:29.788520 14807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:29.788671 14792 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:40:29.788751 14792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "c99b8fb9c301449982c840607b844062"
format_stamp: "Formatted at 2026-06-19 20:40:29 on dist-test-slave-0t40"
server_key: "4adc9b13eba5889887c121d48df08de9"
server_key_iv: "72191a7c775d113966fb0d54dfe76012"
server_key_version: "encryptionkey@0"
I20260619 20:40:29.788803 14792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:29.791857 14792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:29.793874 14792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:29.793977 14792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:29.797574 14792 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38319
I20260619 20:40:29.797621 14859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38319 every 8 connection(s)
I20260619 20:40:29.797919 14792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:29.798206 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 14792
I20260619 20:40:29.798324 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:29.798532 25850 external_mini_cluster.cc:1428] Setting key 60f6b139c18fa2b2adeb0bfea7daa7c3
I20260619 20:40:29.799098 14860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:29.801898 14860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062: Bootstrap starting.
I20260619 20:40:29.802166 14863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:29.799057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.228:53417 (local address 127.0.0.1:38319)
0619 20:40:29.800527 (+ 1470us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:29.800543 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:29.800567 (+ 24us) server_negotiation.cc:407] Connection header received
0619 20:40:29.800951 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:29.800957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:29.800980 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:29.801151 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:29.801755 (+ 604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:29.801757 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:29.801758 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:29.801761 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:29.801796 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:29.801806 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:29.801861 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:29.801955 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 20:40:29.802018 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1422,"thread_start_us":63,"threads_started":1}
I20260619 20:40:29.802595 14860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:29.802906 14860 log.cc:824] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:29.803143 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:29.804636 14860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062: No bootstrap required, opened a new log
I20260619 20:40:29.805557 14863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:29.804470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.228:58439 (local address 127.0.0.1:38319)
0619 20:40:29.804500 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:29.804501 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:29.804624 (+ 123us) server_negotiation.cc:407] Connection header received
0619 20:40:29.804698 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:29.804699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:29.804704 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:29.804757 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:29.805348 (+ 591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:29.805349 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:29.805350 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:29.805354 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:29.805366 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:29.805370 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:29.805426 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:29.805482 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 20:40:29.805495 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 20:40:29.806834 14860 raft_consensus.cc:348] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c99b8fb9c301449982c840607b844062" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38319 } }
I20260619 20:40:29.806968 14860 raft_consensus.cc:374] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:29.806990 14860 raft_consensus.cc:729] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c99b8fb9c301449982c840607b844062, State: Initialized, Role: FOLLOWER
I20260619 20:40:29.807068 14860 consensus_queue.cc:260] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c99b8fb9c301449982c840607b844062" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38319 } }
I20260619 20:40:29.807127 14860 raft_consensus.cc:388] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:29.807180 14860 raft_consensus.cc:482] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:29.807227 14860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:29.807746 14860 raft_consensus.cc:504] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c99b8fb9c301449982c840607b844062" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38319 } }
I20260619 20:40:29.807830 14860 leader_election.cc:302] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c99b8fb9c301449982c840607b844062; no voters:
I20260619 20:40:29.807999 14860 leader_election.cc:288] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:29.808058 14870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:29.808228 14860 sys_catalog.cc:564] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:29.808226 14870 raft_consensus.cc:686] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062 [term 1 LEADER]: Becoming Leader. State: Replica: c99b8fb9c301449982c840607b844062, State: Running, Role: LEADER
I20260619 20:40:29.808368 14870 consensus_queue.cc:237] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c99b8fb9c301449982c840607b844062" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38319 } }
I20260619 20:40:29.808712 14872 sys_catalog.cc:455] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c99b8fb9c301449982c840607b844062. Latest consensus state: current_term: 1 leader_uuid: "c99b8fb9c301449982c840607b844062" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c99b8fb9c301449982c840607b844062" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38319 } } }
I20260619 20:40:29.808799 14872 sys_catalog.cc:458] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:29.809010 14871 sys_catalog.cc:455] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c99b8fb9c301449982c840607b844062" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c99b8fb9c301449982c840607b844062" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38319 } } }
I20260619 20:40:29.809106 14871 sys_catalog.cc:458] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:29.809015 14878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:29.809684 14878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:29.811126 14878 catalog_manager.cc:1347] Generated new cluster ID: b53bacd6ce854335aa71129328e59e23
I20260619 20:40:29.811167 14878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:29.827600 14878 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:29.827791 14878 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:29.830870 14878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c99b8fb9c301449982c840607b844062: Generated new TSK 0
I20260619 20:40:29.831037 14878 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:29.836905 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 14792
2026-06-19T20:40:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:29 dist-test-slave-0t40 krb5kdc[14891](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:29 dist-test-slave-0t40 krb5kdc[14891](info): set up 2 sockets
Jun 19 20:40:29 dist-test-slave-0t40 krb5kdc[14891](info): commencing operation
krb5kdc: starting...
W20260619 20:40:30.869438 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:30 dist-test-slave-0t40 krb5kdc[14891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:40:30.984529 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:39873
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:40005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:39873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:31.065054 14904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:31.065287 14904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:31.065325 14904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:31.066476 14904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:31.066541 14904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:31.066565 14904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:31.066591 14904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:31.066622 14904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:31.068091 14904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:40005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:39873
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:39873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:31.068363 14904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:31.068591 14904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:31.073503 14909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:31.073508 14912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:31.073741 14904 server_base.cc:1034] running on GCE node
W20260619 20:40:31.073496 14910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:31.074070 14904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:31.074357 14904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:31.075521 14904 hybrid_clock.cc:648] HybridClock initialized: now 1781901631075520 us; error 40 us; skew 500 ppm
Jun 19 20:40:31 dist-test-slave-0t40 krb5kdc[14891](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901631, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:31.078004 14904 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:40:31.078456 14904 webserver.cc:460] Webserver started at http://127.25.62.190:34655/ using document root <none> and password file <none>
I20260619 20:40:31.078691 14904 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:31.078758 14904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:31.078910 14904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:31.079907 14904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "528c8fac006542c2a8c3aa0295b9e759"
format_stamp: "Formatted at 2026-06-19 20:40:31 on dist-test-slave-0t40"
server_key: "21c2a3d82f8fbb0613546f6dc1032f5c"
server_key_iv: "5e9e06448eceeda6d8031ac9890f1ae4"
server_key_version: "encryptionkey@0"
I20260619 20:40:31.080255 14904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "528c8fac006542c2a8c3aa0295b9e759"
format_stamp: "Formatted at 2026-06-19 20:40:31 on dist-test-slave-0t40"
server_key: "21c2a3d82f8fbb0613546f6dc1032f5c"
server_key_iv: "5e9e06448eceeda6d8031ac9890f1ae4"
server_key_version: "encryptionkey@0"
I20260619 20:40:31.081610 14904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:40:31.082322 14919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:31.082497 14904 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:40:31.082595 14904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "528c8fac006542c2a8c3aa0295b9e759"
format_stamp: "Formatted at 2026-06-19 20:40:31 on dist-test-slave-0t40"
server_key: "21c2a3d82f8fbb0613546f6dc1032f5c"
server_key_iv: "5e9e06448eceeda6d8031ac9890f1ae4"
server_key_version: "encryptionkey@0"
I20260619 20:40:31.082670 14904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:31.090611 14904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:31.092257 14904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:31.092410 14904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:31.096300 14904 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:39873
I20260619 20:40:31.096333 14971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:39873 every 8 connection(s)
I20260619 20:40:31.096657 14904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:31.097498 14972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:31.099048 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 14904
I20260619 20:40:31.099138 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:31.099331 25850 external_mini_cluster.cc:1428] Setting key 0be889f205a5912c397e4547eb290576
I20260619 20:40:31.099939 14972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759: Bootstrap starting.
I20260619 20:40:31.100493 14972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:31.100780 14972 log.cc:824] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:31.101475 14972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759: No bootstrap required, opened a new log
I20260619 20:40:31.103170 14972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "528c8fac006542c2a8c3aa0295b9e759" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 39873 } }
I20260619 20:40:31.103332 14972 raft_consensus.cc:374] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:31.103422 14972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 528c8fac006542c2a8c3aa0295b9e759, State: Initialized, Role: FOLLOWER
I20260619 20:40:31.103609 14972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "528c8fac006542c2a8c3aa0295b9e759" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 39873 } }
I20260619 20:40:31.103679 14972 raft_consensus.cc:388] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:31.103718 14972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:31.103808 14972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:31.104498 14972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "528c8fac006542c2a8c3aa0295b9e759" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 39873 } }
I20260619 20:40:31.104611 14972 leader_election.cc:302] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 528c8fac006542c2a8c3aa0295b9e759; no voters:
I20260619 20:40:31.104780 14972 leader_election.cc:288] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:31.104835 14977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:31.104995 14977 raft_consensus.cc:686] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759 [term 1 LEADER]: Becoming Leader. State: Replica: 528c8fac006542c2a8c3aa0295b9e759, State: Running, Role: LEADER
I20260619 20:40:31.105072 14972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:31.105170 14977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "528c8fac006542c2a8c3aa0295b9e759" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 39873 } }
I20260619 20:40:31.105445 14977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 528c8fac006542c2a8c3aa0295b9e759. Latest consensus state: current_term: 1 leader_uuid: "528c8fac006542c2a8c3aa0295b9e759" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "528c8fac006542c2a8c3aa0295b9e759" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 39873 } } }
I20260619 20:40:31.105521 14977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:31.105665 14979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "528c8fac006542c2a8c3aa0295b9e759" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "528c8fac006542c2a8c3aa0295b9e759" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 39873 } } }
I20260619 20:40:31.105731 14979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759 [sys.catalog]: This master's current role is: LEADER
Jun 19 20:40:31 dist-test-slave-0t40 krb5kdc[14891](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:40:31.106953 14981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:31.107489 14981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:31.109109 14981 catalog_manager.cc:1347] Generated new cluster ID: 7684fda913f54414bf277751e5c36ecd
I20260619 20:40:31.109161 14981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:31.110831 14975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:31.099812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45471 (local address 127.25.62.190:39873)
0619 20:40:31.099984 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:31.099991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:31.100015 (+ 24us) server_negotiation.cc:407] Connection header received
0619 20:40:31.100455 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:31.100462 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:31.100505 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:31.100654 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:31.101417 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:31.102327 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:31.103091 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:31.103651 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:31.103780 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:31.106910 (+ 3130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:31.106915 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:31.106915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:31.106927 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:31.108929 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:31.109413 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:31.109416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:31.109416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:31.109510 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:31.109706 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:31.109706 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:31.109707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:31.109978 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:31.110115 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:31.110539 (+ 424us) server_negotiation.cc:299] Negotiation successful
0619 20:40:31.110631 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260619 20:40:31.111783 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:31.124210 14975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:31.113188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38049 (local address 127.25.62.190:39873)
0619 20:40:31.113218 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:31.113219 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:31.117183 (+ 3964us) server_negotiation.cc:407] Connection header received
0619 20:40:31.117280 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:31.117281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:31.117292 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:31.117366 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:31.117965 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:31.118759 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:31.119712 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:31.120052 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:31.120123 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:31.121469 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:31.121470 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:31.121471 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:31.121478 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:31.123034 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:31.123421 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:31.123422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:31.123423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:31.123480 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:31.123699 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:31.123700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:31.123700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:31.123822 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:31.123923 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:31.124060 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 20:40:31.124121 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 20:40:31.140393 14981 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:31.140936 14981 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:31.147804 14981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 528c8fac006542c2a8c3aa0295b9e759: Generated new TSK 0
I20260619 20:40:31.148005 14981 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:31.152400 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 14904
W20260619 20:40:31.156800 14889 connection.cc:535] client connection to 127.25.62.190:39873 recv error: Network error: failed to read from TLS socket (remote: 127.25.62.190:39873): Connection reset by peer (error 104)
2026-06-19T20:40:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1316 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:31 dist-test-slave-0t40 krb5kdc[15003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:31 dist-test-slave-0t40 krb5kdc[15003](info): set up 2 sockets
Jun 19 20:40:31 dist-test-slave-0t40 krb5kdc[15003](info): commencing operation
krb5kdc: starting...
W20260619 20:40:32.185499 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:32 dist-test-slave-0t40 krb5kdc[15003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:40:32.302044 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:34553
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:35587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:34553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:32.381377 15016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:32.381564 15016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:32.381587 15016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:32.382721 15016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:32.382761 15016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:32.382776 15016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:32.382789 15016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:32.382843 15016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:32.384255 15016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:35587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:34553
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:34553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:32.384502 15016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:32.384732 15016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:32.389510 15024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:32.389510 15022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:32.389531 15021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:32.389736 15016 server_base.cc:1034] running on GCE node
I20260619 20:40:32.389955 15016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:32.390178 15016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:32.391331 15016 hybrid_clock.cc:648] HybridClock initialized: now 1781901632391322 us; error 32 us; skew 500 ppm
Jun 19 20:40:32 dist-test-slave-0t40 krb5kdc[15003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901632, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:32.396914 15016 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:40:32.397426 15016 webserver.cc:460] Webserver started at http://127.25.62.190:46805/ using document root <none> and password file <none>
I20260619 20:40:32.397663 15016 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:32.397730 15016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:32.397886 15016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:32.398908 15016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "1008d00344034d3a9ccc86c02ebd4c51"
format_stamp: "Formatted at 2026-06-19 20:40:32 on dist-test-slave-0t40"
server_key: "c65c827beeb1543ee1b7afe4a179f182"
server_key_iv: "fd9c260833580c56d2cd375ff99fcf75"
server_key_version: "encryptionkey@0"
I20260619 20:40:32.399260 15016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "1008d00344034d3a9ccc86c02ebd4c51"
format_stamp: "Formatted at 2026-06-19 20:40:32 on dist-test-slave-0t40"
server_key: "c65c827beeb1543ee1b7afe4a179f182"
server_key_iv: "fd9c260833580c56d2cd375ff99fcf75"
server_key_version: "encryptionkey@0"
I20260619 20:40:32.400519 15016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 20:40:32.401333 15031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:32.401500 15016 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 20:40:32.401587 15016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "1008d00344034d3a9ccc86c02ebd4c51"
format_stamp: "Formatted at 2026-06-19 20:40:32 on dist-test-slave-0t40"
server_key: "c65c827beeb1543ee1b7afe4a179f182"
server_key_iv: "fd9c260833580c56d2cd375ff99fcf75"
server_key_version: "encryptionkey@0"
I20260619 20:40:32.401666 15016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:32.419826 15016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:32.421710 15016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:32.421919 15016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:32.425936 15016 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:34553
I20260619 20:40:32.425999 15083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:34553 every 8 connection(s)
I20260619 20:40:32.426348 15016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:32.427175 15084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:32.429333 15084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51: Bootstrap starting.
I20260619 20:40:32.429841 15084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:32.430075 15084 log.cc:824] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:32.430692 15084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51: No bootstrap required, opened a new log
I20260619 20:40:32.432190 15084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1008d00344034d3a9ccc86c02ebd4c51" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 34553 } }
I20260619 20:40:32.432334 15084 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:32.432358 15084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1008d00344034d3a9ccc86c02ebd4c51, State: Initialized, Role: FOLLOWER
I20260619 20:40:32.432456 15084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1008d00344034d3a9ccc86c02ebd4c51" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 34553 } }
I20260619 20:40:32.432509 15084 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:32.432533 15084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:32.432565 15084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:32.433434 15084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1008d00344034d3a9ccc86c02ebd4c51" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 34553 } }
I20260619 20:40:32.433522 15084 leader_election.cc:302] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1008d00344034d3a9ccc86c02ebd4c51; no voters:
I20260619 20:40:32.433645 15084 leader_election.cc:288] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:32.433713 15087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:32.433879 15087 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51 [term 1 LEADER]: Becoming Leader. State: Replica: 1008d00344034d3a9ccc86c02ebd4c51, State: Running, Role: LEADER
I20260619 20:40:32.433899 15084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:32.433967 15087 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1008d00344034d3a9ccc86c02ebd4c51" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 34553 } }
I20260619 20:40:32.434325 15089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1008d00344034d3a9ccc86c02ebd4c51. Latest consensus state: current_term: 1 leader_uuid: "1008d00344034d3a9ccc86c02ebd4c51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1008d00344034d3a9ccc86c02ebd4c51" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 34553 } } }
I20260619 20:40:32.434312 15088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1008d00344034d3a9ccc86c02ebd4c51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1008d00344034d3a9ccc86c02ebd4c51" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 34553 } } }
I20260619 20:40:32.434433 15089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:32.434677 15088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:32.435058 15096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:32.435619 15096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:32.436553 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 15016
I20260619 20:40:32.436664 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:32.436853 25850 external_mini_cluster.cc:1428] Setting key ec76a851c49b7e14cb9d85ce8b53dba8
I20260619 20:40:32.437546 15096 catalog_manager.cc:1347] Generated new cluster ID: 693a5e4765a4400493f067ddf3aa59c6
I20260619 20:40:32.437597 15096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:32.443044 15107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:32.437511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56953 (local address 127.25.62.190:34553)
0619 20:40:32.437668 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:32.437676 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:32.437708 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:40:32.437824 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:32.437832 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:32.437885 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:32.438032 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:32.438237 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:32.439665 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:32.440707 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:32.441412 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:32.441527 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:32.442240 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:32.442244 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:32.442245 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:32.442467 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:32.442512 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:32.442530 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:32.442621 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:32.442800 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 20:40:32.442831 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260619 20:40:32.444039 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:32.452678 15107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:32.445506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59519 (local address 127.25.62.190:34553)
0619 20:40:32.445560 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:32.445561 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:32.449236 (+ 3675us) server_negotiation.cc:407] Connection header received
0619 20:40:32.449305 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:32.449306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:32.449316 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:32.449369 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:32.449567 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:32.450349 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:32.451313 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:32.451642 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:32.451714 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:32.452421 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:32.452422 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:32.452423 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:32.452427 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:32.452443 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:32.452448 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:32.452506 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:32.452587 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 20:40:32.452600 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 20:40:32.476184 15096 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:32.476740 15096 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:32.486650 15096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1008d00344034d3a9ccc86c02ebd4c51: Generated new TSK 0
I20260619 20:40:32.486876 15096 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:32.491544 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 15016
W20260619 20:40:32.496073 15001 connection.cc:535] client connection to 127.25.62.190:34553 recv error: Network error: failed to read from TLS socket (remote: 127.25.62.190:34553): Connection reset by peer (error 104)
2026-06-19T20:40:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1338 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:32 dist-test-slave-0t40 krb5kdc[15115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:32 dist-test-slave-0t40 krb5kdc[15115](info): set up 2 sockets
Jun 19 20:40:32 dist-test-slave-0t40 krb5kdc[15115](info): commencing operation
krb5kdc: starting...
W20260619 20:40:33.529629 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:33 dist-test-slave-0t40 krb5kdc[15115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:40:33.644711 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:38289
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:46173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:38289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:33.724395 15128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:33.724596 15128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:33.724619 15128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:33.725700 15128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:33.725737 15128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:33.725751 15128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:33.725764 15128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:33.725775 15128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:33.727164 15128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:46173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:38289
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:38289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:33.727344 15128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:33.727501 15128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:33.732342 15136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:33.732409 15134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:33.732349 15133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:33.732573 15128 server_base.cc:1034] running on GCE node
I20260619 20:40:33.732795 15128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:33.733129 15128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:33.734304 15128 hybrid_clock.cc:648] HybridClock initialized: now 1781901633734299 us; error 38 us; skew 500 ppm
Jun 19 20:40:33 dist-test-slave-0t40 krb5kdc[15115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901633, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:33.736848 15128 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:40:33.737304 15128 webserver.cc:460] Webserver started at http://127.25.62.190:42633/ using document root <none> and password file <none>
I20260619 20:40:33.737488 15128 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:33.737527 15128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:33.737612 15128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:33.738584 15128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "6efd4a5a69aa4cc783813a2d0b65fc8c"
format_stamp: "Formatted at 2026-06-19 20:40:33 on dist-test-slave-0t40"
server_key: "0462dd09a220a528cf4c070f4fa25624"
server_key_iv: "c14d704c4ca545579f9ec53a80b82df6"
server_key_version: "encryptionkey@0"
I20260619 20:40:33.738943 15128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "6efd4a5a69aa4cc783813a2d0b65fc8c"
format_stamp: "Formatted at 2026-06-19 20:40:33 on dist-test-slave-0t40"
server_key: "0462dd09a220a528cf4c070f4fa25624"
server_key_iv: "c14d704c4ca545579f9ec53a80b82df6"
server_key_version: "encryptionkey@0"
I20260619 20:40:33.740276 15128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 20:40:33.741211 15143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:33.741454 15128 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:33.741562 15128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "6efd4a5a69aa4cc783813a2d0b65fc8c"
format_stamp: "Formatted at 2026-06-19 20:40:33 on dist-test-slave-0t40"
server_key: "0462dd09a220a528cf4c070f4fa25624"
server_key_iv: "c14d704c4ca545579f9ec53a80b82df6"
server_key_version: "encryptionkey@0"
I20260619 20:40:33.741652 15128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:33.744812 15128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:33.746660 15128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:33.746816 15128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:33.750959 15128 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:38289
I20260619 20:40:33.750989 15195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:38289 every 8 connection(s)
I20260619 20:40:33.751312 15128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:33.752308 15196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:33.754710 15196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c: Bootstrap starting.
I20260619 20:40:33.755306 15196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:33.755606 15196 log.cc:824] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:33.756336 15196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c: No bootstrap required, opened a new log
I20260619 20:40:33.757999 15196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6efd4a5a69aa4cc783813a2d0b65fc8c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 38289 } }
I20260619 20:40:33.758131 15196 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:33.758153 15196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6efd4a5a69aa4cc783813a2d0b65fc8c, State: Initialized, Role: FOLLOWER
I20260619 20:40:33.758309 15196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6efd4a5a69aa4cc783813a2d0b65fc8c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 38289 } }
I20260619 20:40:33.758395 15196 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:33.758419 15196 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:33.758466 15196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:33.759222 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 15128
I20260619 20:40:33.759294 15196 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6efd4a5a69aa4cc783813a2d0b65fc8c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 38289 } }
I20260619 20:40:33.759317 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:33.759405 15196 leader_election.cc:302] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6efd4a5a69aa4cc783813a2d0b65fc8c; no voters:
I20260619 20:40:33.759555 25850 external_mini_cluster.cc:1428] Setting key 2e48f723880a8f02e5662d2565887c0e
I20260619 20:40:33.759598 15196 leader_election.cc:288] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:33.759622 15199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:33.759752 15199 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c [term 1 LEADER]: Becoming Leader. State: Replica: 6efd4a5a69aa4cc783813a2d0b65fc8c, State: Running, Role: LEADER
I20260619 20:40:33.759845 15196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:33.759828 15199 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6efd4a5a69aa4cc783813a2d0b65fc8c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 38289 } }
I20260619 20:40:33.760133 15200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6efd4a5a69aa4cc783813a2d0b65fc8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6efd4a5a69aa4cc783813a2d0b65fc8c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 38289 } } }
I20260619 20:40:33.760228 15200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:33.760408 15201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6efd4a5a69aa4cc783813a2d0b65fc8c. Latest consensus state: current_term: 1 leader_uuid: "6efd4a5a69aa4cc783813a2d0b65fc8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6efd4a5a69aa4cc783813a2d0b65fc8c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 38289 } } }
I20260619 20:40:33.760476 15201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:33.760450 15205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:33.761562 15205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:33.762053 15204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:33.760003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59501 (local address 127.25.62.190:38289)
0619 20:40:33.760414 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:33.760423 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:33.760455 (+ 32us) server_negotiation.cc:407] Connection header received
0619 20:40:33.760570 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:33.760577 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:33.760609 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:33.760772 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:33.761578 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:33.761582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:33.761583 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:33.761587 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:33.761626 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:33.761642 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:33.761708 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:33.761809 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 20:40:33.761854 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":64,"threads_started":1}
I20260619 20:40:33.763221 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:33.764608 15205 catalog_manager.cc:1347] Generated new cluster ID: 0c53604a616843c8a6d7968693927ad0
I20260619 20:40:33.764683 15205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:33.766173 15204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:33.764835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53165 (local address 127.25.62.190:38289)
0619 20:40:33.764885 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:33.764886 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:33.765195 (+ 309us) server_negotiation.cc:407] Connection header received
0619 20:40:33.765272 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:33.765273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:33.765298 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:33.765346 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:33.765953 (+ 607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:33.765954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:33.765955 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:33.765959 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:33.765973 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:33.765977 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:33.766033 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:33.766080 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 20:40:33.766094 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 20:40:33.797264 15205 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:33.797472 15205 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:33.799829 15205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6efd4a5a69aa4cc783813a2d0b65fc8c: Generated new TSK 0
I20260619 20:40:33.800009 15205 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:33.808542 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 15128
2026-06-19T20:40:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1316 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:33 dist-test-slave-0t40 krb5kdc[15227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:33 dist-test-slave-0t40 krb5kdc[15227](info): set up 2 sockets
Jun 19 20:40:33 dist-test-slave-0t40 krb5kdc[15227](info): commencing operation
krb5kdc: starting...
W20260619 20:40:34.833549 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:34 dist-test-slave-0t40 krb5kdc[15227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:40:34.951470 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:37685
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:42899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:37685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:35.031234 15240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:35.031440 15240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:35.031463 15240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:35.032618 15240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:35.032661 15240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:35.032675 15240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:35.032688 15240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:35.032730 15240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:35.034220 15240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:42899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:37685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:37685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:35.034487 15240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:35.034722 15240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 20:40:35.039711 15240 server_base.cc:1034] running on GCE node
W20260619 20:40:35.039724 15246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:35.039686 15245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:35.039695 15248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:35.040184 15240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:35.040474 15240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:35.041630 15240 hybrid_clock.cc:648] HybridClock initialized: now 1781901635041614 us; error 48 us; skew 500 ppm
Jun 19 20:40:35 dist-test-slave-0t40 krb5kdc[15227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901635, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:35.043977 15240 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:40:35.044387 15240 webserver.cc:460] Webserver started at http://127.25.62.190:38005/ using document root <none> and password file <none>
I20260619 20:40:35.044616 15240 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:35.044682 15240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:35.044836 15240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:35.045907 15240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "e9860df597f34fbc9bcf30a21e3c139c"
format_stamp: "Formatted at 2026-06-19 20:40:35 on dist-test-slave-0t40"
server_key: "1c480432d39a3f0b37bda42a773614d1"
server_key_iv: "14ac7716e0ff65c81f9d124c87bf5b5a"
server_key_version: "encryptionkey@0"
I20260619 20:40:35.046257 15240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "e9860df597f34fbc9bcf30a21e3c139c"
format_stamp: "Formatted at 2026-06-19 20:40:35 on dist-test-slave-0t40"
server_key: "1c480432d39a3f0b37bda42a773614d1"
server_key_iv: "14ac7716e0ff65c81f9d124c87bf5b5a"
server_key_version: "encryptionkey@0"
I20260619 20:40:35.047529 15240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:40:35.048270 15255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:35.048451 15240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:35.048547 15240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "e9860df597f34fbc9bcf30a21e3c139c"
format_stamp: "Formatted at 2026-06-19 20:40:35 on dist-test-slave-0t40"
server_key: "1c480432d39a3f0b37bda42a773614d1"
server_key_iv: "14ac7716e0ff65c81f9d124c87bf5b5a"
server_key_version: "encryptionkey@0"
I20260619 20:40:35.048628 15240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:35.070468 15240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:35.072433 15240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:35.072599 15240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:35.076586 15240 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:37685
I20260619 20:40:35.076627 15307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:37685 every 8 connection(s)
I20260619 20:40:35.076994 15240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:35.077893 15308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:35.080094 15308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c: Bootstrap starting.
I20260619 20:40:35.080659 15308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:35.080951 15308 log.cc:824] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:35.081657 15308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c: No bootstrap required, opened a new log
I20260619 20:40:35.083181 15308 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9860df597f34fbc9bcf30a21e3c139c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37685 } }
I20260619 20:40:35.083315 15308 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:35.083338 15308 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9860df597f34fbc9bcf30a21e3c139c, State: Initialized, Role: FOLLOWER
I20260619 20:40:35.083501 15308 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9860df597f34fbc9bcf30a21e3c139c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37685 } }
I20260619 20:40:35.083585 15308 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:35.083609 15308 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:35.083636 15308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:35.084440 15308 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9860df597f34fbc9bcf30a21e3c139c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37685 } }
I20260619 20:40:35.084551 15308 leader_election.cc:302] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9860df597f34fbc9bcf30a21e3c139c; no voters:
I20260619 20:40:35.084736 15308 leader_election.cc:288] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:35.084777 15311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:35.084946 15311 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c [term 1 LEADER]: Becoming Leader. State: Replica: e9860df597f34fbc9bcf30a21e3c139c, State: Running, Role: LEADER
I20260619 20:40:35.084995 15308 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:35.085110 15311 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9860df597f34fbc9bcf30a21e3c139c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37685 } }
I20260619 20:40:35.085469 15312 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9860df597f34fbc9bcf30a21e3c139c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9860df597f34fbc9bcf30a21e3c139c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37685 } } }
I20260619 20:40:35.085567 15312 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:35.085484 15313 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9860df597f34fbc9bcf30a21e3c139c. Latest consensus state: current_term: 1 leader_uuid: "e9860df597f34fbc9bcf30a21e3c139c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9860df597f34fbc9bcf30a21e3c139c" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37685 } } }
I20260619 20:40:35.085616 15313 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:35.086100 15319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:35.086354 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 15240
I20260619 20:40:35.086473 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:35.086683 25850 external_mini_cluster.cc:1428] Setting key 36622e18f9b015211d978e005d1c3efb
I20260619 20:40:35.086689 15319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:35.088222 15319 catalog_manager.cc:1347] Generated new cluster ID: 1ef3bf94dae74454a9e989239aa0c8c5
I20260619 20:40:35.088270 15319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 20:40:35 dist-test-slave-0t40 krb5kdc[15227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:40:35.095860 15319 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:35.096477 15319 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:35.099551 15330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:35.087261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48085 (local address 127.25.62.190:37685)
0619 20:40:35.087405 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:35.087415 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:35.087449 (+ 34us) server_negotiation.cc:407] Connection header received
0619 20:40:35.087573 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:35.087579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:35.087656 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:35.087795 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:35.089611 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:35.091061 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:35.092089 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:35.092727 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:35.092845 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:35.095660 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:35.095665 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:35.095665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:35.095678 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:35.097833 (+ 2155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:35.098217 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:35.098220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:35.098220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:35.098337 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:35.098535 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:35.098536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:35.098536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:35.098804 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:35.098964 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:35.099030 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 20:40:35.099301 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":76,"threads_started":1}
I20260619 20:40:35.100639 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:35.102679 15319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9860df597f34fbc9bcf30a21e3c139c: Generated new TSK 0
I20260619 20:40:35.102865 15319 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:35.109778 15330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:35.102436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47119 (local address 127.25.62.190:37685)
0619 20:40:35.102479 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:35.102481 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:35.102495 (+ 14us) server_negotiation.cc:407] Connection header received
0619 20:40:35.102521 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:35.102522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:35.102535 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:35.102612 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:35.103337 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:35.103845 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:35.104635 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:35.104872 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:35.104965 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:35.106380 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:35.106382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:35.106382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:35.106390 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:35.108505 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:35.108917 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:35.108918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:35.108918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:35.108985 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:35.109228 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:35.109229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:35.109229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:35.109350 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:35.109463 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:35.109583 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 20:40:35.109674 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:40:35.112003 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 15240
2026-06-19T20:40:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1302 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:35 dist-test-slave-0t40 krb5kdc[15339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:35 dist-test-slave-0t40 krb5kdc[15339](info): set up 2 sockets
Jun 19 20:40:35 dist-test-slave-0t40 krb5kdc[15339](info): commencing operation
krb5kdc: starting...
W20260619 20:40:36.149590 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:36 dist-test-slave-0t40 krb5kdc[15339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:40:36.265990 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:33071
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:39941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:33071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:36.345541 15352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:36.345737 15352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:36.345762 15352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:36.346910 15352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:36.346957 15352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:36.346971 15352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:36.346987 15352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:36.346998 15352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:36.348452 15352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:39941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:33071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:33071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:36.348701 15352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:36.348924 15352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:36.353555 15358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:36.353542 15360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:36.353551 15357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:36.353833 15352 server_base.cc:1034] running on GCE node
I20260619 20:40:36.354082 15352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:36.354372 15352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:36.355516 15352 hybrid_clock.cc:648] HybridClock initialized: now 1781901636355507 us; error 30 us; skew 500 ppm
Jun 19 20:40:36 dist-test-slave-0t40 krb5kdc[15339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901636, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:36.358055 15352 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:40:36.358522 15352 webserver.cc:460] Webserver started at http://127.25.62.190:35011/ using document root <none> and password file <none>
I20260619 20:40:36.358762 15352 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:36.358829 15352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:36.358989 15352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:36.360039 15352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "990ce5f2aa6d4286a7610e95daa01bec"
format_stamp: "Formatted at 2026-06-19 20:40:36 on dist-test-slave-0t40"
server_key: "9a5c8b87d78abaf43881bf665d9ad09a"
server_key_iv: "f8351550a4aae59fed3b06e612fda5c5"
server_key_version: "encryptionkey@0"
I20260619 20:40:36.360402 15352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "990ce5f2aa6d4286a7610e95daa01bec"
format_stamp: "Formatted at 2026-06-19 20:40:36 on dist-test-slave-0t40"
server_key: "9a5c8b87d78abaf43881bf665d9ad09a"
server_key_iv: "f8351550a4aae59fed3b06e612fda5c5"
server_key_version: "encryptionkey@0"
I20260619 20:40:36.361861 15352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:40:36.362685 15367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:36.362870 15352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:36.362986 15352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "990ce5f2aa6d4286a7610e95daa01bec"
format_stamp: "Formatted at 2026-06-19 20:40:36 on dist-test-slave-0t40"
server_key: "9a5c8b87d78abaf43881bf665d9ad09a"
server_key_iv: "f8351550a4aae59fed3b06e612fda5c5"
server_key_version: "encryptionkey@0"
I20260619 20:40:36.363066 15352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:36.380426 15352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:36.382854 15352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:36.383023 15352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:36.387068 15352 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:33071
I20260619 20:40:36.387116 15419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:33071 every 8 connection(s)
I20260619 20:40:36.387486 15352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:36.388301 15420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:36.390347 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 15352
I20260619 20:40:36.390460 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:36.390564 15420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec: Bootstrap starting.
I20260619 20:40:36.390684 25850 external_mini_cluster.cc:1428] Setting key b076a1adfda090de12ab954c77b0fab0
I20260619 20:40:36.391340 15420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:36.391662 15420 log.cc:824] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:36.392375 15420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec: No bootstrap required, opened a new log
I20260619 20:40:36.394601 15420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990ce5f2aa6d4286a7610e95daa01bec" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 33071 } }
I20260619 20:40:36.394739 15420 raft_consensus.cc:374] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:36.394761 15420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 990ce5f2aa6d4286a7610e95daa01bec, State: Initialized, Role: FOLLOWER
I20260619 20:40:36.394889 15420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990ce5f2aa6d4286a7610e95daa01bec" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 33071 } }
I20260619 20:40:36.394954 15420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:36.394975 15420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:36.395040 15420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:36.395782 15420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990ce5f2aa6d4286a7610e95daa01bec" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 33071 } }
I20260619 20:40:36.395893 15420 leader_election.cc:302] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 990ce5f2aa6d4286a7610e95daa01bec; no voters:
I20260619 20:40:36.396049 15420 leader_election.cc:288] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:36.396070 15425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:36.396181 15425 raft_consensus.cc:686] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec [term 1 LEADER]: Becoming Leader. State: Replica: 990ce5f2aa6d4286a7610e95daa01bec, State: Running, Role: LEADER
I20260619 20:40:36.396312 15420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:36.396317 15425 consensus_queue.cc:237] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990ce5f2aa6d4286a7610e95daa01bec" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 33071 } }
I20260619 20:40:36.396875 15425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 990ce5f2aa6d4286a7610e95daa01bec. Latest consensus state: current_term: 1 leader_uuid: "990ce5f2aa6d4286a7610e95daa01bec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990ce5f2aa6d4286a7610e95daa01bec" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 33071 } } }
I20260619 20:40:36.396965 15425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:36.396914 15427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "990ce5f2aa6d4286a7610e95daa01bec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990ce5f2aa6d4286a7610e95daa01bec" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 33071 } } }
I20260619 20:40:36.397181 15427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:36.397419 15423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:36.391240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33481 (local address 127.25.62.190:33071)
0619 20:40:36.391439 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:36.391447 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:36.391473 (+ 26us) server_negotiation.cc:407] Connection header received
0619 20:40:36.391942 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:36.391949 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:36.392023 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:36.392141 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:36.392368 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:36.393333 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:36.394527 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:36.395147 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:36.395257 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:36.395925 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:36.395929 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:36.395929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:36.396175 (+ 246us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:36.396232 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:36.396250 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:36.396350 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:36.397155 (+ 805us) server_negotiation.cc:299] Negotiation successful
0619 20:40:36.397220 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":101,"threads_started":1}
W20260619 20:40:36.398311 15441 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:40:36.398412 15441 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:40:36.398502 15434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:36.398626 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:36.398664 15434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:36.400362 15434 catalog_manager.cc:1347] Generated new cluster ID: 1361c00ecaeb47c3bc36abcf24ec07f9
I20260619 20:40:36.400414 15434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:36.404094 15423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:36.400176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55015 (local address 127.25.62.190:33071)
0619 20:40:36.400209 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:36.400210 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:36.401111 (+ 901us) server_negotiation.cc:407] Connection header received
0619 20:40:36.401262 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:36.401263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:36.401276 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:36.401343 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:36.401511 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:36.402020 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:36.402782 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:36.403004 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:36.403099 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:36.403830 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:36.403832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:36.403832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:36.403837 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:36.403853 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:36.403858 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:36.403918 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:36.403985 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 20:40:36.404007 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:40:36.423205 15434 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:36.423729 15434 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:36.436264 15434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 990ce5f2aa6d4286a7610e95daa01bec: Generated new TSK 0
I20260619 20:40:36.436493 15434 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:36.439656 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 15352
2026-06-19T20:40:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1328 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:36 dist-test-slave-0t40 krb5kdc[15451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:36 dist-test-slave-0t40 krb5kdc[15451](info): set up 2 sockets
Jun 19 20:40:36 dist-test-slave-0t40 krb5kdc[15451](info): commencing operation
krb5kdc: starting...
W20260619 20:40:37.489425 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:37 dist-test-slave-0t40 krb5kdc[15451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:40:37.607304 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:44987
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:45827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:44987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:37.685573 15464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:37.685776 15464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:37.685808 15464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:37.686954 15464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:37.686995 15464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:37.687011 15464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:37.687023 15464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:37.687036 15464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:37.688483 15464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:45827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:44987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:44987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:37.688725 15464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:37.688958 15464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:37.694012 15470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:37.694018 15472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:37.694070 15464 server_base.cc:1034] running on GCE node
W20260619 20:40:37.694051 15469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:37.694445 15464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:37.694742 15464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:37.695901 15464 hybrid_clock.cc:648] HybridClock initialized: now 1781901637695883 us; error 38 us; skew 500 ppm
Jun 19 20:40:37 dist-test-slave-0t40 krb5kdc[15451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901637, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:37.716964 15464 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:40:37.717481 15464 webserver.cc:460] Webserver started at http://127.25.62.190:45145/ using document root <none> and password file <none>
I20260619 20:40:37.717733 15464 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:37.717803 15464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:37.717963 15464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:37.719041 15464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "d2e28f60e7694b91a94b38ae55d27927"
format_stamp: "Formatted at 2026-06-19 20:40:37 on dist-test-slave-0t40"
server_key: "d44b44181e8a6f1e592ed568ba27de37"
server_key_iv: "aae39e9cf0036d9c37dc3b1b034159fa"
server_key_version: "encryptionkey@0"
I20260619 20:40:37.719439 15464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "d2e28f60e7694b91a94b38ae55d27927"
format_stamp: "Formatted at 2026-06-19 20:40:37 on dist-test-slave-0t40"
server_key: "d44b44181e8a6f1e592ed568ba27de37"
server_key_iv: "aae39e9cf0036d9c37dc3b1b034159fa"
server_key_version: "encryptionkey@0"
I20260619 20:40:37.720863 15464 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:37.721771 15479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:37.721962 15464 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:37.722071 15464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "d2e28f60e7694b91a94b38ae55d27927"
format_stamp: "Formatted at 2026-06-19 20:40:37 on dist-test-slave-0t40"
server_key: "d44b44181e8a6f1e592ed568ba27de37"
server_key_iv: "aae39e9cf0036d9c37dc3b1b034159fa"
server_key_version: "encryptionkey@0"
I20260619 20:40:37.722146 15464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:37.725543 15464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:37.727721 15464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:37.727898 15464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:37.732084 15464 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:44987
I20260619 20:40:37.732187 15531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:44987 every 8 connection(s)
I20260619 20:40:37.732494 15464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:37.733567 15532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:37.735841 15532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927: Bootstrap starting.
I20260619 20:40:37.736433 15532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:37.736723 15532 log.cc:824] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:37.737512 15532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927: No bootstrap required, opened a new log
I20260619 20:40:37.739114 15532 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e28f60e7694b91a94b38ae55d27927" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44987 } }
I20260619 20:40:37.739251 15532 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 20:40:37.739274 15532 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e28f60e7694b91a94b38ae55d27927, State: Initialized, Role: FOLLOWER
I20260619 20:40:37.739431 15532 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e28f60e7694b91a94b38ae55d27927" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44987 } }
I20260619 20:40:37.739494 15532 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 20:40:37.739516 15532 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 20:40:37.739543 15532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:37.740392 15532 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e28f60e7694b91a94b38ae55d27927" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44987 } }
I20260619 20:40:37.740479 15532 leader_election.cc:302] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2e28f60e7694b91a94b38ae55d27927; no voters:
I20260619 20:40:37.740686 15532 leader_election.cc:288] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 20:40:37.740751 15535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:37.740976 15532 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:37.740993 15535 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927 [term 1 LEADER]: Becoming Leader. State: Replica: d2e28f60e7694b91a94b38ae55d27927, State: Running, Role: LEADER
I20260619 20:40:37.741194 15535 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e28f60e7694b91a94b38ae55d27927" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44987 } }
I20260619 20:40:37.741786 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 15464
I20260619 20:40:37.741909 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:37.742127 25850 external_mini_cluster.cc:1428] Setting key fe616e3234a045347304ff42900df41d
I20260619 20:40:37.742144 15537 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2e28f60e7694b91a94b38ae55d27927. Latest consensus state: current_term: 1 leader_uuid: "d2e28f60e7694b91a94b38ae55d27927" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e28f60e7694b91a94b38ae55d27927" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44987 } } }
I20260619 20:40:37.742326 15537 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:37.741570 15536 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2e28f60e7694b91a94b38ae55d27927" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2e28f60e7694b91a94b38ae55d27927" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 44987 } } }
W20260619 20:40:37.742929 15549 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260619 20:40:37.743088 15536 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927 [sys.catalog]: This master's current role is: LEADER
W20260619 20:40:37.743193 15549 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:40:37.743309 15550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:37.743485 15550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:37.744570 15554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:37.742703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53807 (local address 127.25.62.190:44987)
0619 20:40:37.742887 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:37.742897 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:37.742928 (+ 31us) server_negotiation.cc:407] Connection header received
0619 20:40:37.743195 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:37.743203 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:37.743260 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:37.743468 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:37.744138 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:37.744141 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:37.744141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:37.744146 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:37.744189 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:37.744205 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:37.744293 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:37.744351 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 20:40:37.744392 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":102,"threads_started":1}
I20260619 20:40:37.745519 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:37.745549 15550 catalog_manager.cc:1347] Generated new cluster ID: 10aeed5c4ee64a8590a135261a5722ec
I20260619 20:40:37.745589 15550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:37.748018 15554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:37.747073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55641 (local address 127.25.62.190:44987)
0619 20:40:37.747103 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:37.747105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:37.747113 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:40:37.747189 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:37.747190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:37.747194 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:37.747242 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:37.747820 (+ 578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:37.747821 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:37.747821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 20:40:37.747824 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:37.747837 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 20:40:37.747841 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:37.747884 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:37.747948 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 20:40:37.747960 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 20:40:37.760702 15550 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:37.760887 15550 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:37.770573 15550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2e28f60e7694b91a94b38ae55d27927: Generated new TSK 0
I20260619 20:40:37.770763 15550 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:37.778621 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 15464
2026-06-19T20:40:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1338 ms)
[----------] 18 tests from AuthTokenIssuingTest (23723 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:37 dist-test-slave-0t40 krb5kdc[15563](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:37 dist-test-slave-0t40 krb5kdc[15563](info): set up 2 sockets
Jun 19 20:40:37 dist-test-slave-0t40 krb5kdc[15563](info): commencing operation
krb5kdc: starting...
W20260619 20:40:38.821470 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:38 dist-test-slave-0t40 krb5kdc[15563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:40:38.939934 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:35149
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:43149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:35149,127.25.62.189:35045,127.25.62.188:34379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:39.021596 15576 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 20:40:39.021823 15576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:39.021843 15576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:39.021916 15576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:39.023221 15576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:39.023267 15576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:39.023284 15576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:39.023299 15576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:39.023355 15576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:39.024919 15576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:43149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:35149,127.25.62.189:35045,127.25.62.188:34379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:35149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:39.025218 15576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:39.025455 15576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:39.030194 15582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:39.030321 15576 server_base.cc:1034] running on GCE node
W20260619 20:40:39.030201 15584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:39.030189 15581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:39.030607 15576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:39.030840 15576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:39.031976 15576 hybrid_clock.cc:648] HybridClock initialized: now 1781901639031956 us; error 42 us; skew 500 ppm
Jun 19 20:40:39 dist-test-slave-0t40 krb5kdc[15563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901639, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:39.034420 15576 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:40:39.034861 15576 webserver.cc:460] Webserver started at http://127.25.62.190:46797/ using document root <none> and password file <none>
I20260619 20:40:39.035085 15576 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:39.035151 15576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:39.035306 15576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:39.036357 15576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b"
format_stamp: "Formatted at 2026-06-19 20:40:39 on dist-test-slave-0t40"
server_key: "12fbdf33cf2676fc3744d39eb3974569"
server_key_iv: "8cabae9b26e1abc192a34a1d8a5b6fc4"
server_key_version: "encryptionkey@0"
I20260619 20:40:39.036716 15576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b"
format_stamp: "Formatted at 2026-06-19 20:40:39 on dist-test-slave-0t40"
server_key: "12fbdf33cf2676fc3744d39eb3974569"
server_key_iv: "8cabae9b26e1abc192a34a1d8a5b6fc4"
server_key_version: "encryptionkey@0"
I20260619 20:40:39.038137 15576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 20:40:39.038880 15591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:39.039094 15576 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:39.039197 15576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b"
format_stamp: "Formatted at 2026-06-19 20:40:39 on dist-test-slave-0t40"
server_key: "12fbdf33cf2676fc3744d39eb3974569"
server_key_iv: "8cabae9b26e1abc192a34a1d8a5b6fc4"
server_key_version: "encryptionkey@0"
I20260619 20:40:39.039276 15576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:39.073535 15576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:39.075745 15576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:39.075938 15576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:39.079933 15576 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:35149
I20260619 20:40:39.079968 15643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:35149 every 8 connection(s)
I20260619 20:40:39.080333 15576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:39.081343 15644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:39.082885 15644 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:40:39.085366 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 15576
I20260619 20:40:39.085495 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:39.085670 25850 external_mini_cluster.cc:1428] Setting key 38d1f519e50c5cd61d6ef9b499bd6f43
Jun 19 20:40:39 dist-test-slave-0t40 krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901639, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:40:39.093667 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.083602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41512 (local address 127.25.62.190:35149)
0619 20:40:39.083758 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.083762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.083810 (+ 48us) server_negotiation.cc:407] Connection header received
0619 20:40:39.084341 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.084343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.084361 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.084457 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.085097 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.086079 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.087338 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.087859 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.087997 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.090415 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:39.090422 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:39.090422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:39.090428 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:39.091964 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.092429 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.092432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.092432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.092505 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.092780 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.092781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.092781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.093007 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:39.093157 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:39.093402 (+ 245us) server_negotiation.cc:299] Negotiation successful
0619 20:40:39.093508 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
I20260619 20:40:39.093799 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.083528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:35149 (local address 127.0.0.1:41512)
0619 20:40:39.083693 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.083735 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:39.084176 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:39.084497 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:39.084500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:39.084764 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:39.084931 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:39.084935 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.086177 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.086178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:39.087255 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:39.087257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.087993 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.087994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:39.088084 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.088631 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:39.088641 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:39.090322 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:39.092050 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:39.092058 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:39.092061 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:39.092338 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:39.092574 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:39.092575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:39.092575 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:39.092691 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:39.093161 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:39.093169 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:39.093254 (+ 85us) client_negotiation.cc:769] Sending connection context
0619 20:40:39.093371 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 20:40:39.093619 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":70,"thread_start_us":92,"threads_started":1}
WARNING: no policy specified for kudu/127.25.62.189@KRBTEST.COM; defaulting to no policy
I20260619 20:40:39.094542 15644 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:40:39.094906 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.094791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:35045 (local address 127.0.0.1:43056)
0619 20:40:39.094838 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.094887 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.189:35045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 20:40:39.095031 15595 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.62.189:35045: connect: Connection refused (error 111)
I20260619 20:40:39.095448 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.095353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:35045 (local address 127.0.0.1:43058)
0619 20:40:39.095405 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.095428 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.189:35045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 20:40:39.095551 15644 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.189:35045: Network error: Client connection negotiation failed: client connection to 127.25.62.189:35045: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.189.keytab.
Entry for principal kudu/127.25.62.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.189.keytab.
I20260619 20:40:39.112563 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.189:35045
--webserver_interface=127.25.62.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.189.keytab
--principal=kudu/127.25.62.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:43149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:35149,127.25.62.189:35045,127.25.62.188:34379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:40:39.144785 15644 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } attempt: 1
I20260619 20:40:39.145427 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.145262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:35045 (local address 127.0.0.1:43068)
0619 20:40:39.145336 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.145391 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.189:35045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 20:40:39.145852 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.145756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:35045 (local address 127.0.0.1:43076)
0619 20:40:39.145805 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.145833 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.189:35045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 20:40:39.145980 15644 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.189:35045: Network error: Client connection negotiation failed: client connection to 127.25.62.189:35045: connect: Connection refused (error 111)
W20260619 20:40:39.193953 15651 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 20:40:39.194213 15651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:39.194250 15651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:39.194293 15651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:39.195441 15651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:39.195513 15651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:39.195536 15651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:39.195565 15651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:39.195597 15651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:39.197144 15651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:43149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:35149,127.25.62.189:35045,127.25.62.188:34379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.189.keytab
--principal=kudu/127.25.62.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.189:35045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.62.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:39.197403 15651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:39.197638 15651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:39.202391 15657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:39.202420 15656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:39.202600 15651 server_base.cc:1034] running on GCE node
W20260619 20:40:39.202404 15659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:39.202937 15651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:39.203245 15651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:39.204445 15651 hybrid_clock.cc:648] HybridClock initialized: now 1781901639204415 us; error 61 us; skew 500 ppm
Jun 19 20:40:39 dist-test-slave-0t40 krb5kdc[15563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901639, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:39.207005 15651 init.cc:377] Logged in from keytab as kudu/127.25.62.189@KRBTEST.COM (short username kudu)
I20260619 20:40:39.207432 15651 webserver.cc:460] Webserver started at http://127.25.62.189:35395/ using document root <none> and password file <none>
I20260619 20:40:39.207672 15651 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:39.207741 15651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:39.207901 15651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:39.208945 15651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-1/data/instance:
uuid: "5444303212894df2bf0c4acf545428a5"
format_stamp: "Formatted at 2026-06-19 20:40:39 on dist-test-slave-0t40"
server_key: "f5929590da5f3105d8d39a03f4eb3f01"
server_key_iv: "6f7a32c8a7791dd5e336f4fe4aeccf16"
server_key_version: "encryptionkey@0"
I20260619 20:40:39.209368 15651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-1/wal/instance:
uuid: "5444303212894df2bf0c4acf545428a5"
format_stamp: "Formatted at 2026-06-19 20:40:39 on dist-test-slave-0t40"
server_key: "f5929590da5f3105d8d39a03f4eb3f01"
server_key_iv: "6f7a32c8a7791dd5e336f4fe4aeccf16"
server_key_version: "encryptionkey@0"
I20260619 20:40:39.210661 15651 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:39.211382 15666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:39.211562 15651 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 20:40:39.211658 15651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-1/wal
uuid: "5444303212894df2bf0c4acf545428a5"
format_stamp: "Formatted at 2026-06-19 20:40:39 on dist-test-slave-0t40"
server_key: "f5929590da5f3105d8d39a03f4eb3f01"
server_key_iv: "6f7a32c8a7791dd5e336f4fe4aeccf16"
server_key_version: "encryptionkey@0"
I20260619 20:40:39.211742 15651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:39.214169 15644 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } attempt: 2
I20260619 20:40:39.214540 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.214414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:35045 (local address 127.0.0.1:43092)
0619 20:40:39.214481 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.214515 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.189:35045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 20:40:39.214872 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.214759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:35045 (local address 127.0.0.1:43096)
0619 20:40:39.214823 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.214847 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.189:35045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 20:40:39.214960 15644 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.189:35045: Network error: Client connection negotiation failed: client connection to 127.25.62.189:35045: connect: Connection refused (error 111)
I20260619 20:40:39.245301 15651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:39.247210 15651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:39.247397 15651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:39.251403 15651 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.189:35045
I20260619 20:40:39.251483 15718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.189:35045 every 8 connection(s)
I20260619 20:40:39.251828 15651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-1/data/info.pb
I20260619 20:40:39.252810 15719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:39.254426 15719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:40:39.257668 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 15651
I20260619 20:40:39.257738 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-1/wal/instance
I20260619 20:40:39.257910 25850 external_mini_cluster.cc:1428] Setting key dfb8bfbaf0751b2ff2f9b029dec1152b
Jun 19 20:40:39 dist-test-slave-0t40 krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901639, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.189@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:40:39.263528 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.255185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41526 (local address 127.25.62.190:35149)
0619 20:40:39.255264 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.255266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.255424 (+ 158us) server_negotiation.cc:407] Connection header received
0619 20:40:39.255891 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.255892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.255903 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.255958 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.256575 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.257121 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.258290 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.258507 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.258600 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.260680 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:39.260682 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:39.260683 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:39.260691 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:39.262120 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.262567 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.262569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.262569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.262630 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.262879 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.262880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.262881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.262996 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:39.263069 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:39.263351 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 20:40:39.263439 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 20:40:39.263713 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.255113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:35149 (local address 127.0.0.1:41526)
0619 20:40:39.255304 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.255341 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:39.255806 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:39.256047 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:39.256050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:39.256313 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:39.256502 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:39.256505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.257212 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.257213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:39.258151 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:39.258154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.258638 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.258639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:39.258762 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.259275 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:39.259283 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:39.260584 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:39.262249 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:39.262255 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:39.262257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:39.262474 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:39.262698 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:39.262699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:39.262699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:39.262810 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:39.263071 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:39.263078 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:39.263212 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 20:40:39.263319 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 20:40:39.263468 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":97,"threads_started":1}
I20260619 20:40:39.264247 15719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.25.62.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.188@KRBTEST.COM" created.
Jun 19 20:40:39 dist-test-slave-0t40 krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901639, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.189@KRBTEST.COM for kudu/127.25.62.189@KRBTEST.COM
I20260619 20:40:39.272965 15723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.264577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43110 (local address 127.25.62.189:35045)
0619 20:40:39.264717 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.264722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.264789 (+ 67us) server_negotiation.cc:407] Connection header received
0619 20:40:39.264872 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.264874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.264885 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.264945 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.265246 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.266294 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.267215 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.267711 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.267778 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.269847 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:39.269855 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:39.269855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:39.269864 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:39.271623 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.272040 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.272043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.272043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.272102 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.272326 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.272327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.272327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.272460 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:39.272524 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:39.272698 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 20:40:39.272836 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":73,"server-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260619 20:40:39.273072 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.264507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:35045 (local address 127.0.0.1:43110)
0619 20:40:39.264721 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.264732 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:39.264792 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:39.264972 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:39.264973 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:39.265144 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:39.265190 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:39.265192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.266380 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.266381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:39.267141 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:39.267144 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.267795 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.267796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:39.267882 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.268296 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:39.268302 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:39.269768 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:39.271710 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:39.271711 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:39.271711 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:39.271940 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:39.272182 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:39.272183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:39.272184 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:39.272259 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:39.272528 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:39.272530 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:39.272582 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 20:40:39.272669 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 20:40:39.272787 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198}
I20260619 20:40:39.273763 15719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:40:39.274142 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.274006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:34379 (local address 127.0.0.1:35884)
0619 20:40:39.274060 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.274115 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 20:40:39.274331 15669 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.62.188:34379: connect: Connection refused (error 111)
I20260619 20:40:39.274788 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.274682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:34379 (local address 127.0.0.1:35888)
0619 20:40:39.274735 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.274765 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 20:40:39.274907 15719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.188:34379: Network error: Client connection negotiation failed: client connection to 127.25.62.188:34379: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.188.keytab.
Entry for principal kudu/127.25.62.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.188.keytab.
I20260619 20:40:39.284111 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.188:34379
--webserver_interface=127.25.62.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.188.keytab
--principal=kudu/127.25.62.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:43149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:35149,127.25.62.189:35045,127.25.62.188:34379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:40:39.306119 15644 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } attempt: 3
Jun 19 20:40:39 dist-test-slave-0t40 krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901639, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for kudu/127.25.62.189@KRBTEST.COM
I20260619 20:40:39.313375 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.306489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:35045 (local address 127.0.0.1:43114)
0619 20:40:39.306549 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.306568 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:39.306637 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:39.306798 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:39.306798 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:39.306960 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:39.307026 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:39.307028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.307744 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.307745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:39.308271 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:39.308273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.308672 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.308673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:39.308753 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.309244 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:39.309248 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:39.310637 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:39.312315 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:39.312316 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:39.312317 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:39.312486 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:39.312695 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:39.312696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:39.312697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:39.312756 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:39.313002 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:39.313003 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:39.313079 (+ 76us) client_negotiation.cc:769] Sending connection context
0619 20:40:39.313168 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 20:40:39.313271 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 20:40:39.313375 15723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.306564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43114 (local address 127.25.62.189:35045)
0619 20:40:39.306634 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.306637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.306648 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:40:39.306700 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.306700 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.306714 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.306776 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.307107 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.307659 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.308367 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.308597 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.308674 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.310757 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:39.310759 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:39.310759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:39.310773 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:39.312189 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.312566 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.312567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.312568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.312622 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.312820 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.312821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.312821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.312934 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:39.313010 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:39.313192 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 20:40:39.313271 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 20:40:39.313748 15644 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:40:39.314088 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.313989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:34379 (local address 127.0.0.1:35898)
0619 20:40:39.314038 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.314065 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 20:40:39.314509 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.314414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:34379 (local address 127.0.0.1:35908)
0619 20:40:39.314469 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.314493 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 20:40:39.314617 15644 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.188:34379: Network error: Client connection negotiation failed: client connection to 127.25.62.188:34379: connect: Connection refused (error 111)
I20260619 20:40:39.324131 15719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } attempt: 1
I20260619 20:40:39.324623 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.324483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:34379 (local address 127.0.0.1:35916)
0619 20:40:39.324542 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.324589 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 20:40:39.324993 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.324902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:34379 (local address 127.0.0.1:35918)
0619 20:40:39.324948 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.324974 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 20:40:39.325098 15719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.188:34379: Network error: Client connection negotiation failed: client connection to 127.25.62.188:34379: connect: Connection refused (error 111)
I20260619 20:40:39.345743 15644 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } attempt: 1
I20260619 20:40:39.346228 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.346102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:34379 (local address 127.0.0.1:35922)
0619 20:40:39.346162 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.346202 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 20:40:39.346575 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.346497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:34379 (local address 127.0.0.1:35932)
0619 20:40:39.346537 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.346559 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 20:40:39.346691 15644 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.188:34379: Network error: Client connection negotiation failed: client connection to 127.25.62.188:34379: connect: Connection refused (error 111)
W20260619 20:40:39.366161 15726 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 20:40:39.366364 15726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:39.366381 15726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:39.366402 15726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:39.367461 15726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:39.367497 15726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:39.367511 15726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:39.367523 15726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:39.367535 15726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:39.368922 15726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:43149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:35149,127.25.62.189:35045,127.25.62.188:34379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.188.keytab
--principal=kudu/127.25.62.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.188:34379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.62.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:39.369170 15726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:39.369346 15726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:39.374097 15734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:39.374119 15731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:39.374234 15726 server_base.cc:1034] running on GCE node
W20260619 20:40:39.374094 15732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:39.374580 15726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:39.374904 15726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:39.376083 15726 hybrid_clock.cc:648] HybridClock initialized: now 1781901639376073 us; error 34 us; skew 500 ppm
Jun 19 20:40:39 dist-test-slave-0t40 krb5kdc[15563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901639, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:39.378803 15726 init.cc:377] Logged in from keytab as kudu/127.25.62.188@KRBTEST.COM (short username kudu)
I20260619 20:40:39.379272 15726 webserver.cc:460] Webserver started at http://127.25.62.188:37187/ using document root <none> and password file <none>
I20260619 20:40:39.379514 15726 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:39.379586 15726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:39.379750 15726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:39.380841 15726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-2/data/instance:
uuid: "6df89666a657454abb6f4d9ab55b3ba5"
format_stamp: "Formatted at 2026-06-19 20:40:39 on dist-test-slave-0t40"
server_key: "642147ce4ded4e7da8459c5cdda9e2d9"
server_key_iv: "7e24a5767b1a04bee8008301e753e36c"
server_key_version: "encryptionkey@0"
I20260619 20:40:39.381243 15726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-2/wal/instance:
uuid: "6df89666a657454abb6f4d9ab55b3ba5"
format_stamp: "Formatted at 2026-06-19 20:40:39 on dist-test-slave-0t40"
server_key: "642147ce4ded4e7da8459c5cdda9e2d9"
server_key_iv: "7e24a5767b1a04bee8008301e753e36c"
server_key_version: "encryptionkey@0"
I20260619 20:40:39.382628 15726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:40:39.383352 15741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:39.383529 15726 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 20:40:39.383631 15726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-2/wal
uuid: "6df89666a657454abb6f4d9ab55b3ba5"
format_stamp: "Formatted at 2026-06-19 20:40:39 on dist-test-slave-0t40"
server_key: "642147ce4ded4e7da8459c5cdda9e2d9"
server_key_iv: "7e24a5767b1a04bee8008301e753e36c"
server_key_version: "encryptionkey@0"
I20260619 20:40:39.383737 15726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:39.393261 15719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } attempt: 2
I20260619 20:40:39.393684 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.393558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:34379 (local address 127.0.0.1:35938)
0619 20:40:39.393618 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.393657 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 20:40:39.394032 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.393938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:34379 (local address 127.0.0.1:35946)
0619 20:40:39.393982 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.394006 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:34379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 20:40:39.394107 15719 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.188:34379: Network error: Client connection negotiation failed: client connection to 127.25.62.188:34379: connect: Connection refused (error 111)
I20260619 20:40:39.397585 15726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:39.400130 15726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:39.400309 15726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:39.404784 15726 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.188:34379
I20260619 20:40:39.404830 15793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.188:34379 every 8 connection(s)
I20260619 20:40:39.405287 15726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-2/data/info.pb
I20260619 20:40:39.406332 15794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:39.407871 15794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:40:39.408651 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 15726
I20260619 20:40:39.408740 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781901378226483-25850-0/minicluster-data/master-2/wal/instance
I20260619 20:40:39.408916 25850 external_mini_cluster.cc:1428] Setting key 4e0b6de467c76457826fb676f783c8f3
Jun 19 20:40:39 dist-test-slave-0t40 krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901639, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.188@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
Jun 19 20:40:39 dist-test-slave-0t40 krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.188@KRBTEST.COM
I20260619 20:40:39.417549 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.408602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41540 (local address 127.25.62.190:35149)
0619 20:40:39.408653 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.408655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.408926 (+ 271us) server_negotiation.cc:407] Connection header received
0619 20:40:39.409617 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.409619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.409631 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.409687 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.410238 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.410705 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.411603 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.411793 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.411865 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.414353 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:39.414354 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:39.414354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:39.414362 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:39.415917 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.416509 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.416510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.416510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.416560 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.416847 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.416848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.416848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.416948 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:39.416997 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:39.417344 (+ 347us) server_negotiation.cc:299] Negotiation successful
0619 20:40:39.417435 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 20:40:39.417768 15796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.408565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:35149 (local address 127.0.0.1:41540)
0619 20:40:39.408773 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.408821 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:39.409498 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:39.409704 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:39.409708 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:39.409979 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:39.410133 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:39.410138 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.410801 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.410802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:39.411503 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:39.411505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.411904 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.411905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:39.412022 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.412569 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:39.412580 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:39.414247 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:39.416015 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:39.416023 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:39.416027 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:39.416432 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:39.416626 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:39.416627 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:39.416628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:39.416780 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:39.417012 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:39.417017 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:39.417205 (+ 188us) client_negotiation.cc:769] Sending connection context
0619 20:40:39.417324 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 20:40:39.417490 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":100,"threads_started":1}
I20260619 20:40:39.418331 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.409342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35950 (local address 127.25.62.188:34379)
0619 20:40:39.409498 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.409502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.409540 (+ 38us) server_negotiation.cc:407] Connection header received
0619 20:40:39.409658 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.409660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.409684 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.409774 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.410373 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.411157 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.411894 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.412247 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.412326 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.414987 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:39.414995 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:39.414996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:39.415010 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:39.417020 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.417449 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.417452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.417452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.417512 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.417718 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.417719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.417720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.417883 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:39.418011 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:39.418129 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 20:40:39.418209 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260619 20:40:39.418458 15794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:40:39.421847 15644 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } attempt: 2
Jun 19 20:40:39 dist-test-slave-0t40 krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901639, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.188@KRBTEST.COM for kudu/127.25.62.189@KRBTEST.COM
Jun 19 20:40:39 dist-test-slave-0t40 krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:40:39.426406 15796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.418710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:35045 (local address 127.0.0.1:43126)
0619 20:40:39.418821 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.418832 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:39.418878 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:39.419023 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:39.419024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:39.419180 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:39.419233 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:39.419235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.419926 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.419927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:39.420550 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:39.420552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.420897 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.420898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:39.420965 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.421339 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:39.421344 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:39.422781 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:39.425247 (+ 2466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:39.425248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:39.425249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:39.425485 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:39.425709 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:39.425709 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:39.425710 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:39.425765 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:39.426074 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:39.426075 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:39.426114 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 20:40:39.426205 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 20:40:39.426309 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260619 20:40:39.426406 15723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.418775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43126 (local address 127.25.62.189:35045)
0619 20:40:39.418825 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.418826 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.418885 (+ 59us) server_negotiation.cc:407] Connection header received
0619 20:40:39.418932 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.418933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.418943 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.419002 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.419318 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.419845 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.420629 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.420829 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.420880 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.422862 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:39.422863 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:39.422864 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:39.422870 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:39.425187 (+ 2317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.425580 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.425581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.425582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.425635 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.425838 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.425839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.425839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.425974 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:39.426022 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:39.426233 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 20:40:39.426309 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Jun 19 20:40:39 dist-test-slave-0t40 krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901639, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for kudu/127.25.62.188@KRBTEST.COM
I20260619 20:40:39.427423 15794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:40:39.427625 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.419128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41548 (local address 127.25.62.190:35149)
0619 20:40:39.419161 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.419162 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.419170 (+ 8us) server_negotiation.cc:407] Connection header received
0619 20:40:39.419230 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.419231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.419237 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.419296 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.419724 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.420232 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.421344 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.421519 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.421569 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.424234 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:39.424235 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:39.424235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:39.424242 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:39.425890 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.426337 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.426339 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.426339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.426398 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.426592 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.426593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.426593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.426692 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:39.427008 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:39.427420 (+ 412us) server_negotiation.cc:299] Negotiation successful
0619 20:40:39.427530 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 20:40:39.430047 15646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.422095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:34379 (local address 127.0.0.1:35960)
0619 20:40:39.422160 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.422171 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:39.422214 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:39.422489 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:39.422490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:39.422643 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:39.422680 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:39.422681 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.423731 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.423732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:39.424371 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:39.424374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.424923 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.424924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:39.424986 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.425459 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:39.425464 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:39.426973 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:39.428908 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:39.428909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:39.428909 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:39.429173 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:39.429375 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:39.429376 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:39.429376 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:39.429454 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:39.429755 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:39.429756 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:39.429808 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 20:40:39.429855 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 20:40:39.429942 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 20:40:39.430336 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.422295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35960 (local address 127.25.62.188:34379)
0619 20:40:39.422359 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.422361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.422370 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:40:39.422390 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.422391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.422404 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.422470 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.422762 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.423657 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.424496 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.424817 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.424920 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.427061 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:39.427062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:39.427062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:39.427068 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:39.428830 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.429256 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.429257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.429258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.429317 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.429521 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.429522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.429522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.429672 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:39.430165 (+ 493us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:39.430205 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 20:40:39.430280 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":43}
I20260619 20:40:39.432502 15644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b: Bootstrap starting.
Jun 19 20:40:39 dist-test-slave-0t40 krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901639, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.188@KRBTEST.COM for kudu/127.25.62.188@KRBTEST.COM
I20260619 20:40:39.433338 15644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:39.433626 15644 log.cc:824] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b: Log is configured to *not* fsync() on all Append() calls
Jun 19 20:40:39 dist-test-slave-0t40 krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.189@KRBTEST.COM
I20260619 20:40:39.434571 15644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b: No bootstrap required, opened a new log
I20260619 20:40:39.436795 15796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.427691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:34379 (local address 127.0.0.1:35970)
0619 20:40:39.427759 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.427768 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:39.427803 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:39.428376 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:39.428377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:39.428553 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:39.428597 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:39.428599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.429568 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.429569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:39.430093 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:39.430095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.431026 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.431026 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:39.431076 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.431527 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:39.431532 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:39.433301 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:39.435682 (+ 2381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:39.435683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:39.435684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:39.435878 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:39.436088 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:39.436089 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:39.436089 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:39.436162 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:39.436458 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:39.436459 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:39.436499 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 20:40:39.436575 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 20:40:39.436683 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260619 20:40:39.437141 15799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.427953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35970 (local address 127.25.62.188:34379)
0619 20:40:39.428211 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.428214 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.428221 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:40:39.428241 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.428242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.428254 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.428363 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.428672 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.429511 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.430185 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.430941 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.431212 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.433398 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:39.433399 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:39.433400 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:39.433408 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:39.435593 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.435966 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.435967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.435967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.436016 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.436235 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.436236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.436237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.436377 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:39.436464 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:39.436613 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 20:40:39.436732 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":234,"thread_start_us":111,"threads_started":1}
I20260619 20:40:39.437250 15723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.428554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43134 (local address 127.25.62.189:35045)
0619 20:40:39.428593 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.428594 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.428601 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:40:39.428624 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.428625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.428634 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.428692 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.429308 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.429996 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.431413 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.431754 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.431821 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.434476 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:39.434477 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:39.434478 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:39.434485 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:39.435768 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.436170 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.436171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.436171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.436229 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.436418 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.436418 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.436419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.436550 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:39.436716 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:39.436956 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 20:40:39.437031 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 20:40:39.437156 15644 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } } peers { permanent_uuid: "5444303212894df2bf0c4acf545428a5" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } } peers { permanent_uuid: "6df89666a657454abb6f4d9ab55b3ba5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } }
I20260619 20:40:39.437647 15644 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6c526fc7c8404e9fdecdbc6cc9dc4b, State: Initialized, Role: FOLLOWER
I20260619 20:40:39.437760 15644 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } } peers { permanent_uuid: "5444303212894df2bf0c4acf545428a5" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } } peers { permanent_uuid: "6df89666a657454abb6f4d9ab55b3ba5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } }
I20260619 20:40:39.438071 15801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } } peers { permanent_uuid: "5444303212894df2bf0c4acf545428a5" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } } peers { permanent_uuid: "6df89666a657454abb6f4d9ab55b3ba5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } } }
I20260619 20:40:39.438143 15644 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:39.438220 15801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:40:39.438972 15794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5: Bootstrap starting.
I20260619 20:40:39.439019 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:39.439772 15794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:39.440121 15794 log.cc:824] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:39.440084 15618 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41548
I20260619 20:40:39.440232 15618 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 20:40:39.440295 15618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [term 0 FOLLOWER]: Advancing to term 1
W20260619 20:40:39.440130 15812 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260619 20:40:39.441149 15794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5: No bootstrap required, opened a new log
W20260619 20:40:39.441160 15812 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:40:39.441576 15618 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } } peers { permanent_uuid: "5444303212894df2bf0c4acf545428a5" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } } peers { permanent_uuid: "6df89666a657454abb6f4d9ab55b3ba5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } }
I20260619 20:40:39.441831 15618 leader_election.cc:288] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [CANDIDATE]: Term 1 election: Requested vote from peers 5444303212894df2bf0c4acf545428a5 (127.25.62.189:35045), 6df89666a657454abb6f4d9ab55b3ba5 (127.25.62.188:34379)
I20260619 20:40:39.440788 15576 master_runner.cc:186] Error getting master registration for 127.25.62.189:35045: OK, instance_id { permanent_uuid: "5444303212894df2bf0c4acf545428a5" instance_seqno: 1781901639250323 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 20:40:39.443012 15576 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 20:40:39.443845 15794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } } peers { permanent_uuid: "5444303212894df2bf0c4acf545428a5" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } } peers { permanent_uuid: "6df89666a657454abb6f4d9ab55b3ba5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } }
I20260619 20:40:39.444013 15794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6df89666a657454abb6f4d9ab55b3ba5, State: Initialized, Role: FOLLOWER
I20260619 20:40:39.444115 15794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } } peers { permanent_uuid: "5444303212894df2bf0c4acf545428a5" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } } peers { permanent_uuid: "6df89666a657454abb6f4d9ab55b3ba5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } }
I20260619 20:40:39.444443 15794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:39.444485 15768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6df89666a657454abb6f4d9ab55b3ba5"
I20260619 20:40:39.444602 15768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:39.445812 15768 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6c526fc7c8404e9fdecdbc6cc9dc4b in term 1.
I20260619 20:40:39.446050 15594 leader_election.cc:302] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6c526fc7c8404e9fdecdbc6cc9dc4b, 6df89666a657454abb6f4d9ab55b3ba5; no voters:
I20260619 20:40:39.446206 15801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:39.446362 15801 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [term 1 LEADER]: Becoming Leader. State: Replica: 2b6c526fc7c8404e9fdecdbc6cc9dc4b, State: Running, Role: LEADER
I20260619 20:40:39.446463 15801 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } } peers { permanent_uuid: "5444303212894df2bf0c4acf545428a5" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } } peers { permanent_uuid: "6df89666a657454abb6f4d9ab55b3ba5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } }
I20260619 20:40:39.446184 15822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } } peers { permanent_uuid: "5444303212894df2bf0c4acf545428a5" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } } peers { permanent_uuid: "6df89666a657454abb6f4d9ab55b3ba5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } } }
I20260619 20:40:39.447851 15822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:40:39.446904 15834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b6c526fc7c8404e9fdecdbc6cc9dc4b. Latest consensus state: current_term: 1 leader_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } } peers { permanent_uuid: "5444303212894df2bf0c4acf545428a5" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } } peers { permanent_uuid: "6df89666a657454abb6f4d9ab55b3ba5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } } }
I20260619 20:40:39.448225 15834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [sys.catalog]: This master's current role is: LEADER
W20260619 20:40:39.446656 15833 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:40:39.448644 15833 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:40:39.448745 15726 master_runner.cc:186] Error getting master registration for 127.25.62.189:35045: OK, instance_id { permanent_uuid: "5444303212894df2bf0c4acf545428a5" instance_seqno: 1781901639250323 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 20:40:39.450943 15726 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 20:40:39.451973 15799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.443231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35972 (local address 127.25.62.188:34379)
0619 20:40:39.443319 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.443321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.443439 (+ 118us) server_negotiation.cc:407] Connection header received
0619 20:40:39.443549 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.443550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.443562 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.443595 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.444496 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.445325 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.446428 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.446726 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.446780 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.448505 (+ 1725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:39.448507 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:39.448507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:39.448515 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:39.450351 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.450805 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.450807 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.450807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.450869 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.451133 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.451134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.451135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.451356 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:39.451526 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:39.451677 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 20:40:39.451810 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 20:40:39.459578 15647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.443168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41556 (local address 127.25.62.190:35149)
0619 20:40:39.443287 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.443289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.451909 (+ 8620us) server_negotiation.cc:407] Connection header received
0619 20:40:39.452023 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.452024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.452036 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.452070 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.452824 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.453637 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.454597 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.454888 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.454987 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.456431 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:39.456433 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:39.456433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:39.456440 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:39.458268 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.458700 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.458714 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.458714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.458782 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.458991 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.459005 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.459005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.459143 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:39.459252 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:39.459393 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 20:40:39.459481 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260619 20:40:39.460248 15837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:39.460521 15837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:39.463359 15768 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5 [term 1 FOLLOWER]: Refusing update from remote peer 2b6c526fc7c8404e9fdecdbc6cc9dc4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:40:39.463857 15801 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6df89666a657454abb6f4d9ab55b3ba5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:40:39.465530 15822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b6c526fc7c8404e9fdecdbc6cc9dc4b. Latest consensus state: current_term: 1 leader_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } } peers { permanent_uuid: "5444303212894df2bf0c4acf545428a5" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } } peers { permanent_uuid: "6df89666a657454abb6f4d9ab55b3ba5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } } }
I20260619 20:40:39.465637 15822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:40:39.467374 15822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } } peers { permanent_uuid: "5444303212894df2bf0c4acf545428a5" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } } peers { permanent_uuid: "6df89666a657454abb6f4d9ab55b3ba5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } } }
I20260619 20:40:39.467474 15822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:40:39.467648 15837 catalog_manager.cc:1347] Generated new cluster ID: 153b65db55d245e1b2dfb834eb2b506c
I20260619 20:40:39.467702 15837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:39.468348 15801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } } peers { permanent_uuid: "5444303212894df2bf0c4acf545428a5" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } } peers { permanent_uuid: "6df89666a657454abb6f4d9ab55b3ba5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } } }
I20260619 20:40:39.468441 15801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:39.468509 15838 mvcc.cc:205] Tried to move back new op lower bound from 7298669115234336768 to 7298669115173408768. Current Snapshot: MvccSnapshot[applied={T|T < 7298669115234336768 or (T in {7298669115234336768})}]
I20260619 20:40:39.471483 15723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.443502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43148 (local address 127.25.62.189:35045)
0619 20:40:39.443560 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.443562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.459680 (+ 16118us) server_negotiation.cc:407] Connection header received
0619 20:40:39.459805 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.459806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.459818 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.459853 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.460597 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.461531 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.462426 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.462746 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.462816 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.467678 (+ 4862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:39.467680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:39.467681 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:39.467690 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:39.469672 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.470337 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.470338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.470339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.470409 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.470709 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.470710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.470710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.470864 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:39.470935 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:39.471306 (+ 371us) server_negotiation.cc:299] Negotiation successful
0619 20:40:39.471387 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 20:40:39.473496 15839 mvcc.cc:205] Tried to move back new op lower bound from 7298669115234336768 to 7298669115173408768. Current Snapshot: MvccSnapshot[applied={T|T < 7298669115234336768}]
I20260619 20:40:39.484992 15837 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:39.485229 15719 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } attempt: 3
I20260619 20:40:39.489399 15837 catalog_manager.cc:1504] Loading token signing keys...
Jun 19 20:40:39 dist-test-slave-0t40 krb5kdc[15563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901639, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.189@KRBTEST.COM for kudu/127.25.62.188@KRBTEST.COM
I20260619 20:40:39.493623 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.485473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:34379 (local address 127.0.0.1:35980)
0619 20:40:39.485556 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:39.485567 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:39.485605 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:39.485867 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:39.485868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:39.486060 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:39.486109 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:39.486112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.487107 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.487108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:39.487879 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:39.487881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.488426 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.488427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:39.488481 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.488970 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:39.488975 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:39.490553 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:39.492440 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:39.492441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:39.492442 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:39.492657 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:39.492905 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:39.492905 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:39.492906 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:39.492976 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:39.493313 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:39.493314 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:39.493373 (+ 59us) client_negotiation.cc:769] Sending connection context
0619 20:40:39.493432 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 20:40:39.493522 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 20:40:39.494235 15799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:39.485688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35980 (local address 127.25.62.188:34379)
0619 20:40:39.485728 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.485730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.485737 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:40:39.485758 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.485759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.485770 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.485820 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.486226 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.486961 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.487998 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:39.488294 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:39.488356 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:39.490645 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:39.490647 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:39.490647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:39.490654 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:39.492329 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.492746 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.492747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.492748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.492822 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:39.493075 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:39.493076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:39.493077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:39.493213 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:39.493271 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:39.494069 (+ 798us) server_negotiation.cc:299] Negotiation successful
0619 20:40:39.494147 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:40:39.495852 15719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5: Bootstrap starting.
I20260619 20:40:39.496606 15719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:39.496927 15719 log.cc:824] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:39.498050 15719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5: No bootstrap required, opened a new log
I20260619 20:40:39.501494 15719 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } } peers { permanent_uuid: "5444303212894df2bf0c4acf545428a5" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } } peers { permanent_uuid: "6df89666a657454abb6f4d9ab55b3ba5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } }
I20260619 20:40:39.501806 15719 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5444303212894df2bf0c4acf545428a5, State: Initialized, Role: FOLLOWER
I20260619 20:40:39.501871 15837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b: Generated new TSK 0
I20260619 20:40:39.502061 15837 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 20:40:39.502048 15719 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } } peers { permanent_uuid: "5444303212894df2bf0c4acf545428a5" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } } peers { permanent_uuid: "6df89666a657454abb6f4d9ab55b3ba5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } }
I20260619 20:40:39.502691 15693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5444303212894df2bf0c4acf545428a5"
I20260619 20:40:39.502617 15844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } } peers { permanent_uuid: "5444303212894df2bf0c4acf545428a5" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } } peers { permanent_uuid: "6df89666a657454abb6f4d9ab55b3ba5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } } }
I20260619 20:40:39.502900 15844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:40:39.503059 15719 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:39.503153 15692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:39.504098 15692 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5 [term 1 FOLLOWER]: Refusing update from remote peer 2b6c526fc7c8404e9fdecdbc6cc9dc4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:40:39.505096 15834 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5444303212894df2bf0c4acf545428a5" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 20:40:39.505262 15855 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:40:39.505306 15855 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:40:39.508096 15693 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b6c526fc7c8404e9fdecdbc6cc9dc4b in term 1.
I20260619 20:40:39.509451 15844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b6c526fc7c8404e9fdecdbc6cc9dc4b. Latest consensus state: current_term: 1 leader_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } } peers { permanent_uuid: "5444303212894df2bf0c4acf545428a5" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } } peers { permanent_uuid: "6df89666a657454abb6f4d9ab55b3ba5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } } }
I20260619 20:40:39.509588 15844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:40:39.509660 15844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } } peers { permanent_uuid: "5444303212894df2bf0c4acf545428a5" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } } peers { permanent_uuid: "6df89666a657454abb6f4d9ab55b3ba5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } } }
I20260619 20:40:39.509842 15844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:40:39.518240 15858 mvcc.cc:205] Tried to move back new op lower bound from 7298669115393441792 to 7298669115173408768. Current Snapshot: MvccSnapshot[applied={T|T < 7298669115234336768}]
W20260619 20:40:39.529500 15647 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:39.527957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41564 (local address 127.25.62.190:35149)
0619 20:40:39.528001 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.528003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.528842 (+ 839us) server_negotiation.cc:407] Connection header received
0619 20:40:39.528961 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.528962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.528975 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.529008 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.529431 (+ 423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41564: BlockingRecv error: recv got EOF from 127.0.0.1:41564 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 20:40:39.530359 15723 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:39.528305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43152 (local address 127.25.62.189:35045)
0619 20:40:39.528342 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.528344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.529868 (+ 1524us) server_negotiation.cc:407] Connection header received
0619 20:40:39.529976 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.529977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.529987 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.530015 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.530304 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43152: BlockingRecv error: recv got EOF from 127.0.0.1:43152 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 20:40:39.531172 15799 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:39.528601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35982 (local address 127.25.62.188:34379)
0619 20:40:39.528654 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.528656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.530683 (+ 2027us) server_negotiation.cc:407] Connection header received
0619 20:40:39.530777 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.530779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.530788 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.530816 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.531111 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35982: BlockingRecv error: recv got EOF from 127.0.0.1:35982 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260619 20:40:39.533540 15723 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:39.532775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43156 (local address 127.25.62.189:35045)
0619 20:40:39.532806 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.532807 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.533062 (+ 255us) server_negotiation.cc:407] Connection header received
0619 20:40:39.533166 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.533167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.533183 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.533211 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.533499 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43156: BlockingRecv error: recv got EOF from 127.0.0.1:43156 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 20:40:39.536477 15723 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:39.535755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43160 (local address 127.25.62.189:35045)
0619 20:40:39.535784 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.535786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.536028 (+ 242us) server_negotiation.cc:407] Connection header received
0619 20:40:39.536117 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.536118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.536127 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.536154 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.536436 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43160: BlockingRecv error: recv got EOF from 127.0.0.1:43160 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 20:40:39.540614 15723 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:39.539865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43176 (local address 127.25.62.189:35045)
0619 20:40:39.539897 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.539898 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.540153 (+ 255us) server_negotiation.cc:407] Connection header received
0619 20:40:39.540243 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.540244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.540253 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.540280 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.540573 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43176: BlockingRecv error: recv got EOF from 127.0.0.1:43176 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260619 20:40:39.547142 15723 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:39.546348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43188 (local address 127.25.62.189:35045)
0619 20:40:39.546384 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.546386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.546639 (+ 253us) server_negotiation.cc:407] Connection header received
0619 20:40:39.546732 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.546733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.546743 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.546771 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.547096 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43188: BlockingRecv error: recv got EOF from 127.0.0.1:43188 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 20:40:39.557353 15723 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:39.556387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43200 (local address 127.25.62.189:35045)
0619 20:40:39.556429 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.556431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.556702 (+ 271us) server_negotiation.cc:407] Connection header received
0619 20:40:39.556808 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.556809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.556821 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.556851 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.557284 (+ 433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43200: BlockingRecv error: recv got EOF from 127.0.0.1:43200 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 20:40:39.575738 15723 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:39.574816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43216 (local address 127.25.62.189:35045)
0619 20:40:39.574858 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.574861 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.575145 (+ 284us) server_negotiation.cc:407] Connection header received
0619 20:40:39.575252 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.575253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.575265 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.575296 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.575677 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43216: BlockingRecv error: recv got EOF from 127.0.0.1:43216 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 20:40:39.610298 15723 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:39.609327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43232 (local address 127.25.62.189:35045)
0619 20:40:39.609375 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.609377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.609678 (+ 301us) server_negotiation.cc:407] Connection header received
0619 20:40:39.609785 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.609787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.609799 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.609828 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.610238 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43232: BlockingRecv error: recv got EOF from 127.0.0.1:43232 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 20:40:39.676455 15723 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:39.675770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43238 (local address 127.25.62.189:35045)
0619 20:40:39.675824 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.675826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.675970 (+ 144us) server_negotiation.cc:407] Connection header received
0619 20:40:39.676001 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.676002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.676013 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.676070 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.676394 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43238: BlockingRecv error: recv got EOF from 127.0.0.1:43238 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260619 20:40:39.806602 15723 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:39.805816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43242 (local address 127.25.62.189:35045)
0619 20:40:39.805900 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:39.805903 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:39.806023 (+ 120us) server_negotiation.cc:407] Connection header received
0619 20:40:39.806126 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:39.806127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:39.806147 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:39.806223 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:39.806555 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43242: BlockingRecv error: recv got EOF from 127.0.0.1:43242 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 20:40:40.038420 15917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b6c526fc7c8404e9fdecdbc6cc9dc4b" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 35149 } } peers { permanent_uuid: "5444303212894df2bf0c4acf545428a5" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 35045 } } peers { permanent_uuid: "6df89666a657454abb6f4d9ab55b3ba5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 34379 } } }
I20260619 20:40:40.038589 15917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b6c526fc7c8404e9fdecdbc6cc9dc4b [sys.catalog]: This master's current role is: LEADER
W20260619 20:40:40.064671 15723 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:40.064003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43256 (local address 127.25.62.189:35045)
0619 20:40:40.064072 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:40.064074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:40.064084 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:40:40.064139 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:40.064141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:40.064151 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:40.064204 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:40.064587 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43256: BlockingRecv error: recv got EOF from 127.0.0.1:43256 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 20:40:40.449102 15833 catalog_manager.cc:1259] Loaded cluster ID: 153b65db55d245e1b2dfb834eb2b506c
I20260619 20:40:40.449177 15833 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5: loading cluster ID for follower catalog manager: success
I20260619 20:40:40.449934 15833 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5: acquiring CA information for follower catalog manager: success
I20260619 20:40:40.450173 15833 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6df89666a657454abb6f4d9ab55b3ba5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:40:40.505736 15855 catalog_manager.cc:1259] Loaded cluster ID: 153b65db55d245e1b2dfb834eb2b506c
I20260619 20:40:40.505980 15855 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5: loading cluster ID for follower catalog manager: success
I20260619 20:40:40.506922 15855 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5: acquiring CA information for follower catalog manager: success
I20260619 20:40:40.507267 15855 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5444303212894df2bf0c4acf545428a5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:40:40.580895 15929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:40.577987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43270 (local address 127.25.62.189:35045)
0619 20:40:40.578123 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:40.578126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:40.578220 (+ 94us) server_negotiation.cc:407] Connection header received
0619 20:40:40.578301 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:40.578302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:40.578370 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:40.578460 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:40:40.578627 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:40.579133 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:40.579973 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:40.580172 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:40.580233 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:40.580435 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:40:40.580602 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:40:40.580706 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:40.580772 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 20:40:40.580796 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":53,"threads_started":1}
I20260619 20:40:40.585032 15935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:40.582759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35992 (local address 127.25.62.188:34379)
0619 20:40:40.582848 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:40.582850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:40.582914 (+ 64us) server_negotiation.cc:407] Connection header received
0619 20:40:40.582987 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:40.582988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:40.583022 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:40.583087 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:40:40.583149 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:40.583611 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:40.584281 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:40.584472 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:40.584524 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:40.584643 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:40:40.584735 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:40:40.584805 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:40.584892 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 20:40:40.584910 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69,"thread_start_us":47,"threads_started":1}
I20260619 20:40:40.589283 15935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:40.586608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36002 (local address 127.25.62.188:34379)
0619 20:40:40.586681 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:40.586683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:40.586795 (+ 112us) server_negotiation.cc:407] Connection header received
0619 20:40:40.586869 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:40.586870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:40.586880 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:40.586924 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:40:40.587020 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:40.587636 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:40.588501 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:40.588799 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:40.588863 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:40.589030 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:40:40.589080 (+ 50us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:40:40.589132 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:40.589180 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 20:40:40.589204 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 20:40:40.589639 15929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:40.586842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43272 (local address 127.25.62.189:35045)
0619 20:40:40.586889 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:40.586891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:40.587044 (+ 153us) server_negotiation.cc:407] Connection header received
0619 20:40:40.587110 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:40.587111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:40.587122 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:40.587175 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:40:40.587312 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:40.588043 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:40.588861 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:40.589152 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:40.589208 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:40.589371 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:40:40.589402 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:40:40.589448 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:40.589534 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 20:40:40.589557 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 20:40:40.590282 15940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:40.586587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41572 (local address 127.25.62.190:35149)
0619 20:40:40.586728 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:40.586730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:40.586958 (+ 228us) server_negotiation.cc:407] Connection header received
0619 20:40:40.587099 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:40.587101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:40.587147 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:40.587224 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:40:40.587418 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:40.588182 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:40.589221 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:40.589560 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:40.589599 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:40.589858 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:40:40.589971 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:40:40.590007 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:40.590158 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 20:40:40.590188 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":54,"threads_started":1}
I20260619 20:40:40.592135 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 15576
I20260619 20:40:40.597193 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 15651
I20260619 20:40:40.602113 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 15726
2026-06-19T20:40:40Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2825 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 20:40:40 dist-test-slave-0t40 krb5kdc[15946](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 20:40:40 dist-test-slave-0t40 krb5kdc[15946](info): set up 2 sockets
Jun 19 20:40:40 dist-test-slave-0t40 krb5kdc[15946](info): commencing operation
krb5kdc: starting...
W20260619 20:40:41.657629 25850 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/test-user.keytab.
Jun 19 20:40:41 dist-test-slave-0t40 krb5kdc[15946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T20:40:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T20:40:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.25.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
Entry for principal kudu/127.25.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab.
I20260619 20:40:41.780383 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:37963
--webserver_interface=127.25.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:37963,127.25.62.189:40021,127.25.62.188:44995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 20:40:41.859642 15959 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 20:40:41.859870 15959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:41.859899 15959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:41.859951 15959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:41.861064 15959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:41.861127 15959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:41.861151 15959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:41.861186 15959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:41.861212 15959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:41.862641 15959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:37963,127.25.62.189:40021,127.25.62.188:44995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.190.keytab
--principal=kudu/127.25.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.190:37963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.25.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:41.862905 15959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:41.863123 15959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:41.867794 15967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:41.867892 15965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:41.867975 15959 server_base.cc:1034] running on GCE node
W20260619 20:40:41.868047 15964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:41.868266 15959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:41.868530 15959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:41.869676 15959 hybrid_clock.cc:648] HybridClock initialized: now 1781901641869654 us; error 38 us; skew 500 ppm
Jun 19 20:40:41 dist-test-slave-0t40 krb5kdc[15946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901641, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:41.872067 15959 init.cc:377] Logged in from keytab as kudu/127.25.62.190@KRBTEST.COM (short username kudu)
I20260619 20:40:41.872481 15959 webserver.cc:460] Webserver started at http://127.25.62.190:33851/ using document root <none> and password file <none>
I20260619 20:40:41.872700 15959 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:41.872766 15959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:41.872915 15959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:41.874014 15959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-0/data/instance:
uuid: "66a40fff02cf4633b7d99a18ef9c0288"
format_stamp: "Formatted at 2026-06-19 20:40:41 on dist-test-slave-0t40"
server_key: "443731dbd5b6d17cc133b272fa7586d1"
server_key_iv: "b56e6253c840eae3247917a9013b6300"
server_key_version: "encryptionkey@0"
I20260619 20:40:41.874374 15959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-0/wal/instance:
uuid: "66a40fff02cf4633b7d99a18ef9c0288"
format_stamp: "Formatted at 2026-06-19 20:40:41 on dist-test-slave-0t40"
server_key: "443731dbd5b6d17cc133b272fa7586d1"
server_key_iv: "b56e6253c840eae3247917a9013b6300"
server_key_version: "encryptionkey@0"
I20260619 20:40:41.875697 15959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 20:40:41.876530 15974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:41.876739 15959 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 20:40:41.876860 15959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-0/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-0/wal
uuid: "66a40fff02cf4633b7d99a18ef9c0288"
format_stamp: "Formatted at 2026-06-19 20:40:41 on dist-test-slave-0t40"
server_key: "443731dbd5b6d17cc133b272fa7586d1"
server_key_iv: "b56e6253c840eae3247917a9013b6300"
server_key_version: "encryptionkey@0"
I20260619 20:40:41.876950 15959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:41.894531 15959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:41.896349 15959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:41.896520 15959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:41.900628 15959 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.190:37963
I20260619 20:40:41.900673 16026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.190:37963 every 8 connection(s)
I20260619 20:40:41.901098 15959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-0/data/info.pb
I20260619 20:40:41.902048 16027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:41.903496 16027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:40:41.904872 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 15959
I20260619 20:40:41.904984 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-0/wal/instance
I20260619 20:40:41.905208 25850 external_mini_cluster.cc:1428] Setting key 6e1d1bf1ff9cfb56eb199858d05facfb
Jun 19 20:40:41 dist-test-slave-0t40 krb5kdc[15946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901641, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.25.62.189@KRBTEST.COM; defaulting to no policy
I20260619 20:40:41.913515 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:41.904115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37963 (local address 127.0.0.1:50162)
0619 20:40:41.904280 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:41.904323 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:41.904897 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:41.905240 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:41.905243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:41.905618 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:41.905867 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:41.905872 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:41.907166 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:41.907167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:41.907839 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:41.907841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:41.908361 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:41.908362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:41.908427 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:41.908963 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:41.908970 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:41.910341 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:41.912057 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:41.912063 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:41.912065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:41.912252 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:41.912498 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:41.912499 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:41.912499 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:41.912586 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:41.912924 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:41.912929 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:41.912986 (+ 57us) client_negotiation.cc:769] Sending connection context
0619 20:40:41.913113 (+ 127us) client_negotiation.cc:240] Negotiation successful
0619 20:40:41.913335 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":91,"thread_start_us":92,"threads_started":1}
I20260619 20:40:41.913519 16030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:41.904313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50162 (local address 127.25.62.190:37963)
0619 20:40:41.904466 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:41.904471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:41.904511 (+ 40us) server_negotiation.cc:407] Connection header received
0619 20:40:41.905102 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:41.905104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:41.905121 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:41.905201 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:41.905965 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:41.907068 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:41.907945 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:41.908244 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:41.908379 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:41.910441 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:41.910448 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:41.910448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:41.910455 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:41.911960 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:41.912351 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:41.912354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:41.912354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:41.912431 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:41.912652 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:41.912653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:41.912654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:41.912855 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:41.912944 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:41.913145 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 20:40:41.913275 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260619 20:40:41.914258 16027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:40:41.914584 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:41.914484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:40021 (local address 127.0.0.1:46584)
0619 20:40:41.914528 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:41.914566 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 20:40:41.914707 15978 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40021: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.189@KRBTEST.COM" created.
I20260619 20:40:41.915165 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:41.915027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:40021 (local address 127.0.0.1:46590)
0619 20:40:41.915104 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:41.915133 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260619 20:40:41.915324 16027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.189:40021: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40021: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.189.keytab.
Entry for principal kudu/127.25.62.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.189.keytab.
I20260619 20:40:41.930195 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.189:40021
--webserver_interface=127.25.62.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.189.keytab
--principal=kudu/127.25.62.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:37963,127.25.62.189:40021,127.25.62.188:44995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:40:41.964591 16027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } attempt: 1
I20260619 20:40:41.965165 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:41.965017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:40021 (local address 127.0.0.1:46598)
0619 20:40:41.965093 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:41.965133 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 20:40:41.965641 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:41.965542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:40021 (local address 127.0.0.1:46614)
0619 20:40:41.965594 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:41.965622 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 20:40:41.965781 16027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.189:40021: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40021: connect: Connection refused (error 111)
W20260619 20:40:42.009586 16034 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 20:40:42.009828 16034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:42.009857 16034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:42.009903 16034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:42.011042 16034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:42.011109 16034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:42.011135 16034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:42.011165 16034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:42.011191 16034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:42.012706 16034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:37963,127.25.62.189:40021,127.25.62.188:44995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.189.keytab
--principal=kudu/127.25.62.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.189:40021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.25.62.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:42.012977 16034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:42.013235 16034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:42.017973 16042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:42.017982 16039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:42.017982 16040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:42.018234 16034 server_base.cc:1034] running on GCE node
I20260619 20:40:42.018471 16034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:42.018847 16034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:42.020100 16034 hybrid_clock.cc:648] HybridClock initialized: now 1781901642020075 us; error 40 us; skew 500 ppm
Jun 19 20:40:42 dist-test-slave-0t40 krb5kdc[15946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901642, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:42.022663 16034 init.cc:377] Logged in from keytab as kudu/127.25.62.189@KRBTEST.COM (short username kudu)
I20260619 20:40:42.023150 16034 webserver.cc:460] Webserver started at http://127.25.62.189:42209/ using document root <none> and password file <none>
I20260619 20:40:42.023393 16034 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:42.023460 16034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:42.023594 16034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:42.024609 16034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-1/data/instance:
uuid: "04c12347f09245f2aa0028b6a81c5ba9"
format_stamp: "Formatted at 2026-06-19 20:40:42 on dist-test-slave-0t40"
server_key: "1c84245406180f0f0892f770fb8b0571"
server_key_iv: "615d0c07cf28ab537382ed183eaeefbc"
server_key_version: "encryptionkey@0"
I20260619 20:40:42.024971 16034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-1/wal/instance:
uuid: "04c12347f09245f2aa0028b6a81c5ba9"
format_stamp: "Formatted at 2026-06-19 20:40:42 on dist-test-slave-0t40"
server_key: "1c84245406180f0f0892f770fb8b0571"
server_key_iv: "615d0c07cf28ab537382ed183eaeefbc"
server_key_version: "encryptionkey@0"
I20260619 20:40:42.026289 16034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:40:42.027046 16049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:42.027252 16034 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 20:40:42.027349 16034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-1/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-1/wal
uuid: "04c12347f09245f2aa0028b6a81c5ba9"
format_stamp: "Formatted at 2026-06-19 20:40:42 on dist-test-slave-0t40"
server_key: "1c84245406180f0f0892f770fb8b0571"
server_key_iv: "615d0c07cf28ab537382ed183eaeefbc"
server_key_version: "encryptionkey@0"
I20260619 20:40:42.027436 16034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:42.033946 16027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } attempt: 2
I20260619 20:40:42.034391 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.034272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:40021 (local address 127.0.0.1:46622)
0619 20:40:42.034333 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.034365 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 20:40:42.034729 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.034637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:40021 (local address 127.0.0.1:46630)
0619 20:40:42.034679 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.034711 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260619 20:40:42.034818 16027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.189:40021: Network error: Client connection negotiation failed: client connection to 127.25.62.189:40021: connect: Connection refused (error 111)
I20260619 20:40:42.042302 16034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:42.044523 16034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:42.044677 16034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:42.048652 16034 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.189:40021
I20260619 20:40:42.048684 16101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.189:40021 every 8 connection(s)
I20260619 20:40:42.049054 16034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-1/data/info.pb
I20260619 20:40:42.050046 16102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:42.051539 16102 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:40:42.054572 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 16034
I20260619 20:40:42.054687 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-1/wal/instance
I20260619 20:40:42.054900 25850 external_mini_cluster.cc:1428] Setting key 36ae0e7e2c32252522b8dd5ad1a12f5b
Jun 19 20:40:42 dist-test-slave-0t40 krb5kdc[15946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901642, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.189@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:40:42.061187 16030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.052194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50176 (local address 127.25.62.190:37963)
0619 20:40:42.052255 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.052257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.052478 (+ 221us) server_negotiation.cc:407] Connection header received
0619 20:40:42.052903 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.052904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.052914 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.052967 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.053584 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.054055 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.055458 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.055646 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.055702 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.057910 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:42.057913 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:42.057913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:42.057921 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:42.059781 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.060154 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.060156 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.060156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.060228 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.060474 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.060475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.060475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.060631 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:42.060725 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:42.060939 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 20:40:42.061081 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 20:40:42.061201 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.052183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37963 (local address 127.0.0.1:50176)
0619 20:40:42.052363 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.052397 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:42.052821 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:42.053028 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:42.053031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:42.053304 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:42.053501 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:42.053505 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.054132 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.054133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:42.055382 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:42.055385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.055739 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.055740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:42.055900 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.056483 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:42.056496 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:42.057805 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:42.059874 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:42.059880 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:42.059881 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:42.060081 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:42.060308 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:42.060309 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:42.060309 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:42.060409 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:42.060703 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:42.060707 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:42.060806 (+ 99us) client_negotiation.cc:769] Sending connection context
0619 20:40:42.060887 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 20:40:42.060992 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
I20260619 20:40:42.061870 16102 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.25.62.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.25.62.188@KRBTEST.COM" created.
Jun 19 20:40:42 dist-test-slave-0t40 krb5kdc[15946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901642, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.189@KRBTEST.COM for kudu/127.25.62.189@KRBTEST.COM
I20260619 20:40:42.069494 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.062218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46634 (local address 127.25.62.189:40021)
0619 20:40:42.062388 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.062392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.062407 (+ 15us) server_negotiation.cc:407] Connection header received
0619 20:40:42.062423 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.062424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.062434 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.062516 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.062759 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.063474 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.064129 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.064522 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.064615 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.066464 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:42.066472 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:42.066473 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:42.066480 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:42.068101 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.068545 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.068548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.068548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.068597 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.068816 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.068816 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.068817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.068940 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:42.068996 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:42.069241 (+ 245us) server_negotiation.cc:299] Negotiation successful
0619 20:40:42.069364 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":148,"thread_start_us":93,"threads_started":1}
I20260619 20:40:42.069506 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.062144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:40021 (local address 127.0.0.1:46634)
0619 20:40:42.062204 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.062213 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:42.062250 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:42.062515 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:42.062516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:42.062649 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:42.062687 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:42.062689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.063553 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.063554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:42.064038 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:42.064039 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.064615 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.064616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:42.064680 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.065117 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:42.065128 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:42.066385 (+ 1257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:42.068197 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:42.068198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:42.068199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:42.068461 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:42.068667 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:42.068668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:42.068668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:42.068746 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:42.069010 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:42.069012 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:42.069088 (+ 76us) client_negotiation.cc:769] Sending connection context
0619 20:40:42.069180 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 20:40:42.069311 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 20:40:42.070240 16102 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:40:42.070627 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.070452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:44995 (local address 127.0.0.1:35316)
0619 20:40:42.070504 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.070560 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 20:40:42.070816 16050 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.25.62.188:44995: connect: Connection refused (error 111)
I20260619 20:40:42.071310 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.071195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:44995 (local address 127.0.0.1:35318)
0619 20:40:42.071251 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.071284 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 20:40:42.071450 16102 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.188:44995: Network error: Client connection negotiation failed: client connection to 127.25.62.188:44995: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.25.62.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.188.keytab.
Entry for principal kudu/127.25.62.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.188.keytab.
I20260619 20:40:42.084375 25850 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXT8PDl/build/release/bin/kudu
/tmp/dist-test-taskXT8PDl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.188:44995
--webserver_interface=127.25.62.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.188.keytab
--principal=kudu/127.25.62.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.25.62.148:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.25.62.190:37963,127.25.62.189:40021,127.25.62.188:44995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 20:40:42.120700 16102 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } attempt: 1
I20260619 20:40:42.121270 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.121107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:44995 (local address 127.0.0.1:35324)
0619 20:40:42.121175 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.121232 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 20:40:42.121640 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.121548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:44995 (local address 127.0.0.1:35336)
0619 20:40:42.121594 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.121621 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 20:40:42.121766 16102 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.188:44995: Network error: Client connection negotiation failed: client connection to 127.25.62.188:44995: connect: Connection refused (error 111)
I20260619 20:40:42.126017 16027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } attempt: 3
Jun 19 20:40:42 dist-test-slave-0t40 krb5kdc[15946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901641, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for kudu/127.25.62.189@KRBTEST.COM
I20260619 20:40:42.132964 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.126334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:40021 (local address 127.0.0.1:46640)
0619 20:40:42.126394 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.126404 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:42.126471 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:42.126606 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:42.126607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:42.126781 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:42.126848 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:42.126851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.127511 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.127513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:42.128133 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:42.128135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.128532 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.128533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:42.128614 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.129086 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:42.129090 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:42.130412 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:42.131962 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:42.131963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:42.131963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:42.132129 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:42.132339 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:42.132340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:42.132340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:42.132402 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:42.132643 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:42.132644 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:42.132689 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 20:40:42.132778 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 20:40:42.132874 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 20:40:42.132962 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.126391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46640 (local address 127.25.62.189:40021)
0619 20:40:42.126453 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.126455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.126466 (+ 11us) server_negotiation.cc:407] Connection header received
0619 20:40:42.126532 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.126533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.126545 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.126600 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.126931 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.127425 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.128212 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.128444 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.128531 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.130506 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:42.130507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:42.130507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:42.130515 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:42.131865 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.132199 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.132200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.132200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.132263 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.132461 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.132462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.132462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.132573 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:42.132651 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:42.132803 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 20:40:42.132873 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 20:40:42.133466 16027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:40:42.133785 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.133686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:44995 (local address 127.0.0.1:35346)
0619 20:40:42.133738 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.133763 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 20:40:42.134147 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.134069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:44995 (local address 127.0.0.1:35356)
0619 20:40:42.134112 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.134134 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 20:40:42.134245 16027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.188:44995: Network error: Client connection negotiation failed: client connection to 127.25.62.188:44995: connect: Connection refused (error 111)
I20260619 20:40:42.165387 16027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } attempt: 1
I20260619 20:40:42.165829 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.165686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:44995 (local address 127.0.0.1:35372)
0619 20:40:42.165752 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.165804 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 20:40:42.166154 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.166062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:44995 (local address 127.0.0.1:35384)
0619 20:40:42.166108 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.166132 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 20:40:42.166242 16027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.188:44995: Network error: Client connection negotiation failed: client connection to 127.25.62.188:44995: connect: Connection refused (error 111)
W20260619 20:40:42.166445 16109 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 20:40:42.166640 16109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 20:40:42.166697 16109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 20:40:42.166733 16109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 20:40:42.167924 16109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 20:40:42.167968 16109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 20:40:42.167982 16109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 20:40:42.167996 16109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 20:40:42.168008 16109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 20:40:42.169569 16109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.25.62.148:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.25.62.190:37963,127.25.62.189:40021,127.25.62.188:44995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/krb5kdc/kudu_127.25.62.188.keytab
--principal=kudu/127.25.62.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.25.62.188:44995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.25.62.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 19:22:44 UTC on ccycloud.quasar-pzqoph.root.comops.site
build id 4456546
I20260619 20:40:42.169818 16109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 20:40:42.170042 16109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 20:40:42.174746 16115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:42.174784 16114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 20:40:42.174757 16117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 20:40:42.174955 16109 server_base.cc:1034] running on GCE node
I20260619 20:40:42.175184 16109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 20:40:42.175464 16109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 20:40:42.176631 16109 hybrid_clock.cc:648] HybridClock initialized: now 1781901642176618 us; error 29 us; skew 500 ppm
Jun 19 20:40:42 dist-test-slave-0t40 krb5kdc[15946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901642, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 20:40:42.179184 16109 init.cc:377] Logged in from keytab as kudu/127.25.62.188@KRBTEST.COM (short username kudu)
I20260619 20:40:42.179754 16109 webserver.cc:460] Webserver started at http://127.25.62.188:35773/ using document root <none> and password file <none>
I20260619 20:40:42.179999 16109 fs_manager.cc:359] Metadata directory not provided
I20260619 20:40:42.180065 16109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 20:40:42.180197 16109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 20:40:42.181381 16109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-2/data/instance:
uuid: "314acf4d1bd041259ec49bf76e561ce5"
format_stamp: "Formatted at 2026-06-19 20:40:42 on dist-test-slave-0t40"
server_key: "e16f9a02303c9419c0ca82723bfd1fdf"
server_key_iv: "4762136574930897b039a35831967795"
server_key_version: "encryptionkey@0"
I20260619 20:40:42.181734 16109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-2/wal/instance:
uuid: "314acf4d1bd041259ec49bf76e561ce5"
format_stamp: "Formatted at 2026-06-19 20:40:42 on dist-test-slave-0t40"
server_key: "e16f9a02303c9419c0ca82723bfd1fdf"
server_key_iv: "4762136574930897b039a35831967795"
server_key_version: "encryptionkey@0"
I20260619 20:40:42.183015 16109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 20:40:42.183724 16124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:42.183903 16109 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 20:40:42.183998 16109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-2/data,/tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-2/wal
uuid: "314acf4d1bd041259ec49bf76e561ce5"
format_stamp: "Formatted at 2026-06-19 20:40:42 on dist-test-slave-0t40"
server_key: "e16f9a02303c9419c0ca82723bfd1fdf"
server_key_iv: "4762136574930897b039a35831967795"
server_key_version: "encryptionkey@0"
I20260619 20:40:42.184077 16109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 20:40:42.189924 16102 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } attempt: 2
I20260619 20:40:42.190315 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.190185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:44995 (local address 127.0.0.1:35386)
0619 20:40:42.190245 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.190282 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 20:40:42.190665 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.190572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:44995 (local address 127.0.0.1:35398)
0619 20:40:42.190617 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.190642 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.25.62.188:44995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 20:40:42.190788 16102 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.25.62.188:44995: Network error: Client connection negotiation failed: client connection to 127.25.62.188:44995: connect: Connection refused (error 111)
I20260619 20:40:42.208034 16109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 20:40:42.210299 16109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 20:40:42.210453 16109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 20:40:42.214527 16109 rpc_server.cc:307] RPC server started. Bound to: 127.25.62.188:44995
I20260619 20:40:42.214560 16176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.25.62.188:44995 every 8 connection(s)
I20260619 20:40:42.214897 16109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-2/data/info.pb
I20260619 20:40:42.215757 16177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 20:40:42.217276 16177 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:40:42.219184 25850 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXT8PDl/build/release/bin/kudu as pid 16109
I20260619 20:40:42.219316 25850 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXT8PDl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781901378226483-25850-0/minicluster-data/master-2/wal/instance
I20260619 20:40:42.219554 25850 external_mini_cluster.cc:1428] Setting key cb45b0281a16be33eae0a85811d735f5
Jun 19 20:40:42 dist-test-slave-0t40 krb5kdc[15946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901642, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.188@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
Jun 19 20:40:42 dist-test-slave-0t40 krb5kdc[15946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.188@KRBTEST.COM
I20260619 20:40:42.227166 16030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.218021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50190 (local address 127.25.62.190:37963)
0619 20:40:42.218110 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.218112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.218216 (+ 104us) server_negotiation.cc:407] Connection header received
0619 20:40:42.218703 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.218704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.218713 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.218755 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.219415 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.220079 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.221511 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.221726 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.221821 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.224191 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:42.224193 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:42.224193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:42.224201 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:42.225693 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.226218 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.226219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.226220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.226283 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.226576 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.226576 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.226577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.226680 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:42.226728 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:42.227045 (+ 317us) server_negotiation.cc:299] Negotiation successful
0619 20:40:42.227104 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 20:40:42.227458 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.217916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.190:37963 (local address 127.0.0.1:50190)
0619 20:40:42.218098 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.218136 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:42.218605 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:42.218810 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:42.218813 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:42.219063 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:42.219268 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:42.219272 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.220175 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.220176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:42.221406 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:42.221408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.221848 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.221849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:42.221975 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.222527 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:42.222535 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:42.224091 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:42.225772 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:42.225781 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:42.225783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:42.226142 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:42.226352 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:42.226353 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:42.226353 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:42.226497 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:42.226745 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:42.226750 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:42.226898 (+ 148us) client_negotiation.cc:769] Sending connection context
0619 20:40:42.227019 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 20:40:42.227212 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260619 20:40:42.228032 16177 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:40:42.228688 16181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.219944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35408 (local address 127.25.62.188:44995)
0619 20:40:42.220197 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.220200 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.220214 (+ 14us) server_negotiation.cc:407] Connection header received
0619 20:40:42.220232 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.220233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.220248 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.220301 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.221100 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.221778 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.222540 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.222829 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.222907 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.225164 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:42.225172 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:42.225173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:42.225180 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:42.227197 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.227621 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.227624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.227624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.227695 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.227930 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.227931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.227931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.228110 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:42.228254 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:42.228477 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 20:40:42.228578 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":88,"threads_started":1}
Jun 19 20:40:42 dist-test-slave-0t40 krb5kdc[15946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901642, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.188@KRBTEST.COM for kudu/127.25.62.189@KRBTEST.COM
Jun 19 20:40:42 dist-test-slave-0t40 krb5kdc[15946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.190@KRBTEST.COM
I20260619 20:40:42.235240 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.228254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.189:40021 (local address 127.0.0.1:46656)
0619 20:40:42.228307 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.228319 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:42.228364 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:42.228509 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:42.228510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:42.228718 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:42.228784 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:42.228786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.229668 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.229669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:42.230517 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:42.230520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.231011 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.231011 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:42.231064 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.231479 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:42.231484 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:42.232717 (+ 1233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:42.234229 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:42.234230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:42.234230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:42.234387 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:42.234613 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:42.234614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:42.234615 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:42.234671 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:42.234930 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:42.234931 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:42.234972 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 20:40:42.235056 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 20:40:42.235144 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 20:40:42.235235 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.228345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46656 (local address 127.25.62.189:40021)
0619 20:40:42.228394 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.228396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.228405 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:40:42.228432 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.228433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.228443 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.228506 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.228874 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.229570 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.230647 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.230933 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.231013 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.232800 (+ 1787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:42.232801 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:42.232802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:42.232809 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:42.234105 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.234488 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.234489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.234490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.234547 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.234748 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.234749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.234749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.234854 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:42.234926 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:42.235081 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 20:40:42.235149 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 20:40:42.235549 16177 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } has no permanent_uuid. Determining permanent_uuid...
I20260619 20:40:42.237397 16030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.229891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50202 (local address 127.25.62.190:37963)
0619 20:40:42.229931 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.229933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.229987 (+ 54us) server_negotiation.cc:407] Connection header received
0619 20:40:42.230055 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.230055 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.230063 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.230147 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.230687 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.231341 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.232056 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.232230 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.232283 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.234589 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:42.234591 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:42.234591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:42.234597 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:42.236407 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.236699 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.236699 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.236700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.236755 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.236896 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.236897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.236897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.237019 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:42.237194 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:42.237280 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 20:40:42.237341 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Jun 19 20:40:42 dist-test-slave-0t40 krb5kdc[15946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901642, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.188@KRBTEST.COM for kudu/127.25.62.188@KRBTEST.COM
I20260619 20:40:42.241377 16027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } attempt: 2
Jun 19 20:40:42 dist-test-slave-0t40 krb5kdc[15946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.25.62.189@KRBTEST.COM
I20260619 20:40:42.242444 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.235774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:44995 (local address 127.0.0.1:35418)
0619 20:40:42.235856 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.235865 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:42.235910 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:42.236019 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:42.236020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:42.236120 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:42.236145 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:42.236146 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.236937 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.236938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:42.237692 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:42.237695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.238092 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.238093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:42.238154 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.238459 (+ 305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:42.238465 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:42.239597 (+ 1132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:42.241364 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:42.241365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:42.241366 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:42.241584 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:42.241772 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:42.241772 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:42.241773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:42.241825 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:42.242153 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:42.242153 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:42.242192 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 20:40:42.242265 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 20:40:42.242354 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260619 20:40:42.242697 16181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.235821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35418 (local address 127.25.62.188:44995)
0619 20:40:42.235856 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.235858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.235901 (+ 43us) server_negotiation.cc:407] Connection header received
0619 20:40:42.235947 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.235947 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.235957 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.236019 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.236212 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.236864 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.237765 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.238014 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.238087 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.239692 (+ 1605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:42.239694 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:42.239694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:42.239700 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:42.241284 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.241662 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.241663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.241663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.241708 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.241895 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.241896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.241896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.242028 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:42.242158 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:42.242557 (+ 399us) server_negotiation.cc:299] Negotiation successful
0619 20:40:42.242625 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:40:42.244510 16177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5: Bootstrap starting.
I20260619 20:40:42.245344 16177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:42.245687 16177 log.cc:824] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5: Log is configured to *not* fsync() on all Append() calls
Jun 19 20:40:42 dist-test-slave-0t40 krb5kdc[15946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901641, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.190@KRBTEST.COM for kudu/127.25.62.188@KRBTEST.COM
I20260619 20:40:42.246518 16177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5: No bootstrap required, opened a new log
I20260619 20:40:42.247933 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.237998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46660 (local address 127.25.62.189:40021)
0619 20:40:42.238038 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.238039 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.238046 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:40:42.238066 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.238067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.238077 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.238136 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.238799 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.239224 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.239953 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.240141 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.240221 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.242515 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:42.242516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:42.242517 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:42.242524 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:42.246627 (+ 4103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.247008 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.247009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.247010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.247065 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.247253 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.247254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.247254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.247397 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:42.247567 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:42.247757 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 20:40:42.247836 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:40:42.248569 16177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a40fff02cf4633b7d99a18ef9c0288" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } } peers { permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } } peers { permanent_uuid: "314acf4d1bd041259ec49bf76e561ce5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } }
I20260619 20:40:42.248734 16177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 314acf4d1bd041259ec49bf76e561ce5, State: Initialized, Role: FOLLOWER
I20260619 20:40:42.248824 16177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a40fff02cf4633b7d99a18ef9c0288" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } } peers { permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } } peers { permanent_uuid: "314acf4d1bd041259ec49bf76e561ce5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } }
I20260619 20:40:42.249094 16184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a40fff02cf4633b7d99a18ef9c0288" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } } peers { permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } } peers { permanent_uuid: "314acf4d1bd041259ec49bf76e561ce5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } } }
I20260619 20:40:42.249153 16177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:42.249161 16184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:40:42.249470 25850 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 20:40:42.249897 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.241629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:44995 (local address 127.0.0.1:35424)
0619 20:40:42.241674 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.241684 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:42.241720 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:42.241962 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:42.241963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:42.242135 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:42.242176 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:42.242178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.243562 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.243564 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:42.244300 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:42.244303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.244696 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.244697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:42.244751 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.245180 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:42.245185 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:42.246511 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:42.248793 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:42.248794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:42.248795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:42.249020 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:42.249265 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:42.249266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:42.249266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:42.249342 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:42.249644 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:42.249645 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:42.249687 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 20:40:42.249731 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 20:40:42.249814 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 20:40:42.250355 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.241667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35424 (local address 127.25.62.188:44995)
0619 20:40:42.241810 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.241812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.241818 (+ 6us) server_negotiation.cc:407] Connection header received
0619 20:40:42.241835 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.241836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.241846 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.241941 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.242238 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.243504 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.244395 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.244588 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.244693 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.246649 (+ 1956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:42.246650 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:42.246651 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:42.246657 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:42.248701 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.249121 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.249122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.249122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.249184 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.249415 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.249416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.249417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.249566 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:42.250083 (+ 517us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:42.250135 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 20:40:42.250217 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":99,"threads_started":1}
I20260619 20:40:42.250396 16001 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "66a40fff02cf4633b7d99a18ef9c0288"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50202
W20260619 20:40:42.251247 16195 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:40:42.251638 16195 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:40:42.251852 16027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288: Bootstrap starting.
I20260619 20:40:42.252540 16027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:42.252847 16027 log.cc:824] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:42.253635 16027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288: No bootstrap required, opened a new log
I20260619 20:40:42.254952 16027 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a40fff02cf4633b7d99a18ef9c0288" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } } peers { permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } } peers { permanent_uuid: "314acf4d1bd041259ec49bf76e561ce5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } }
I20260619 20:40:42.255080 16027 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a40fff02cf4633b7d99a18ef9c0288, State: Initialized, Role: FOLLOWER
I20260619 20:40:42.255149 16027 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a40fff02cf4633b7d99a18ef9c0288" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } } peers { permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } } peers { permanent_uuid: "314acf4d1bd041259ec49bf76e561ce5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } }
I20260619 20:40:42.255470 16027 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:42.255494 16001 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 20:40:42.255427 16198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a40fff02cf4633b7d99a18ef9c0288" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } } peers { permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } } peers { permanent_uuid: "314acf4d1bd041259ec49bf76e561ce5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } } }
I20260619 20:40:42.255544 16001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:42.256350 16001 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a40fff02cf4633b7d99a18ef9c0288" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } } peers { permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } } peers { permanent_uuid: "314acf4d1bd041259ec49bf76e561ce5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } }
I20260619 20:40:42.256575 16001 leader_election.cc:288] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [CANDIDATE]: Term 1 election: Requested vote from peers 04c12347f09245f2aa0028b6a81c5ba9 (127.25.62.189:40021), 314acf4d1bd041259ec49bf76e561ce5 (127.25.62.188:44995)
I20260619 20:40:42.257041 16151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "66a40fff02cf4633b7d99a18ef9c0288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "314acf4d1bd041259ec49bf76e561ce5"
I20260619 20:40:42.257176 16151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:42.257598 16198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:40:42.257990 16151 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a40fff02cf4633b7d99a18ef9c0288 in term 1.
I20260619 20:40:42.257084 16109 master_runner.cc:186] Error getting master registration for 127.25.62.190:37963: OK, instance_id { permanent_uuid: "66a40fff02cf4633b7d99a18ef9c0288" instance_seqno: 1781901641899472 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 20:40:42.260147 16109 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 20:40:42.258203 15975 leader_election.cc:302] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 314acf4d1bd041259ec49bf76e561ce5, 66a40fff02cf4633b7d99a18ef9c0288; no voters:
I20260619 20:40:42.261329 15959 master_runner.cc:186] Error getting master registration for 127.25.62.189:40021: OK, instance_id { permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" instance_seqno: 1781901642047525 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 20:40:42.264508 16198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 20:40:42.264719 16198 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [term 1 LEADER]: Becoming Leader. State: Replica: 66a40fff02cf4633b7d99a18ef9c0288, State: Running, Role: LEADER
I20260619 20:40:42.264807 16198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a40fff02cf4633b7d99a18ef9c0288" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } } peers { permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } } peers { permanent_uuid: "314acf4d1bd041259ec49bf76e561ce5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } }
I20260619 20:40:42.265354 16198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66a40fff02cf4633b7d99a18ef9c0288. Latest consensus state: current_term: 1 leader_uuid: "66a40fff02cf4633b7d99a18ef9c0288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a40fff02cf4633b7d99a18ef9c0288" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } } peers { permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } } peers { permanent_uuid: "314acf4d1bd041259ec49bf76e561ce5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } } }
I20260619 20:40:42.265476 16198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:42.264359 15959 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 20:40:42.265955 16219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 20:40:42.266322 16030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.257925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50214 (local address 127.25.62.190:37963)
0619 20:40:42.257964 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.257966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.258617 (+ 651us) server_negotiation.cc:407] Connection header received
0619 20:40:42.258721 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.258722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.258735 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.258768 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.259607 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.260281 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.261338 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.261622 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.261675 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.263392 (+ 1717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:42.263393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:42.263394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:42.263400 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:42.264988 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.265430 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.265431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.265432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.265488 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.265715 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.265716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.265716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.265847 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:42.265930 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:42.266153 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 20:40:42.266232 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 20:40:42.266887 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.258441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46674 (local address 127.25.62.189:40021)
0619 20:40:42.258485 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.258487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.258987 (+ 500us) server_negotiation.cc:407] Connection header received
0619 20:40:42.259084 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.259085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.259095 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.259124 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.259842 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.260543 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.261659 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.261944 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.261985 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.263481 (+ 1496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:42.263482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:42.263483 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:42.263489 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:42.265157 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.265751 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.265753 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.265753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.265807 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.266006 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.266007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.266007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.266121 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:42.266185 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:42.266706 (+ 521us) server_negotiation.cc:299] Negotiation successful
0619 20:40:42.266796 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 20:40:42.267457 16219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 20:40:42.269587 16151 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5 [term 1 FOLLOWER]: Refusing update from remote peer 66a40fff02cf4633b7d99a18ef9c0288: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:40:42.269932 16217 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "314acf4d1bd041259ec49bf76e561ce5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 20:40:42.271281 16184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66a40fff02cf4633b7d99a18ef9c0288. Latest consensus state: current_term: 1 leader_uuid: "66a40fff02cf4633b7d99a18ef9c0288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a40fff02cf4633b7d99a18ef9c0288" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } } peers { permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } } peers { permanent_uuid: "314acf4d1bd041259ec49bf76e561ce5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } } }
I20260619 20:40:42.271384 16184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:40:42.277114 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.258717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35440 (local address 127.25.62.188:44995)
0619 20:40:42.258762 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.258764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.266835 (+ 8071us) server_negotiation.cc:407] Connection header received
0619 20:40:42.267015 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.267016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.267028 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.267059 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.267926 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.268643 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.269927 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.270266 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.270334 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.272854 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:42.272855 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:42.272856 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:42.272863 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:42.274616 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.275290 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.275292 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.275292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.275381 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.275674 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.275675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.275675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.275808 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:42.276757 (+ 949us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:42.276833 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 20:40:42.276968 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:40:42.277798 16217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "66a40fff02cf4633b7d99a18ef9c0288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a40fff02cf4633b7d99a18ef9c0288" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } } peers { permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } } peers { permanent_uuid: "314acf4d1bd041259ec49bf76e561ce5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } } }
I20260619 20:40:42.277892 16217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:42.278286 16219 catalog_manager.cc:1347] Generated new cluster ID: e5294225f26941bab02d17a677095ca8
I20260619 20:40:42.278333 16219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 20:40:42.279237 16221 mvcc.cc:205] Tried to move back new op lower bound from 7298669126733479936 to 7298669126717353984. Current Snapshot: MvccSnapshot[applied={T|T < 7298669126733479936 or (T in {7298669126733479936})}]
I20260619 20:40:42.279218 16184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "66a40fff02cf4633b7d99a18ef9c0288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a40fff02cf4633b7d99a18ef9c0288" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } } peers { permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } } peers { permanent_uuid: "314acf4d1bd041259ec49bf76e561ce5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } } }
I20260619 20:40:42.279300 16184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:40:42.281953 16102 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } attempt: 3
Jun 19 20:40:42 dist-test-slave-0t40 krb5kdc[15946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781901642, etypes {rep=17 tkt=17 ses=17}, kudu/127.25.62.189@KRBTEST.COM for kudu/127.25.62.188@KRBTEST.COM
I20260619 20:40:42.290732 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.282197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.25.62.188:44995 (local address 127.0.0.1:35446)
0619 20:40:42.282271 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0619 20:40:42.282281 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 20:40:42.282316 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 20:40:42.282625 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 20:40:42.282626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 20:40:42.282787 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 20:40:42.282830 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:42.282832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.283780 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.283781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:42.284541 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 20:40:42.284544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.285078 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.285079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 20:40:42.285135 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.285608 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 20:40:42.285613 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 20:40:42.287917 (+ 2304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 20:40:42.289736 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:42.289737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:42.289738 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:42.289941 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:42.290131 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 20:40:42.290132 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 20:40:42.290132 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 20:40:42.290182 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 20:40:42.290448 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 20:40:42.290449 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 20:40:42.290491 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 20:40:42.290550 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 20:40:42.290635 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 20:40:42.291062 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:42.282474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35446 (local address 127.25.62.188:44995)
0619 20:40:42.282507 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.282509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.282515 (+ 6us) server_negotiation.cc:407] Connection header received
0619 20:40:42.282532 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.282533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.282541 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.282584 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.282908 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.283629 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.284636 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:42.284942 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:42.284999 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:42.288006 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 20:40:42.288008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 20:40:42.288008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 20:40:42.288015 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 20:40:42.289646 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.290016 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.290017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.290018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.290068 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 20:40:42.290241 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 20:40:42.290242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 20:40:42.290242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 20:40:42.290366 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 20:40:42.290423 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:42.290909 (+ 486us) server_negotiation.cc:299] Negotiation successful
0619 20:40:42.290979 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 20:40:42.292464 16102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9: Bootstrap starting.
I20260619 20:40:42.293193 16102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9: Neither blocks nor log segments found. Creating new log.
I20260619 20:40:42.293504 16102 log.cc:824] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9: Log is configured to *not* fsync() on all Append() calls
I20260619 20:40:42.297676 16102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9: No bootstrap required, opened a new log
I20260619 20:40:42.299695 16102 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a40fff02cf4633b7d99a18ef9c0288" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } } peers { permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } } peers { permanent_uuid: "314acf4d1bd041259ec49bf76e561ce5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } }
I20260619 20:40:42.299834 16102 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04c12347f09245f2aa0028b6a81c5ba9, State: Initialized, Role: FOLLOWER
I20260619 20:40:42.299914 16102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a40fff02cf4633b7d99a18ef9c0288" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } } peers { permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } } peers { permanent_uuid: "314acf4d1bd041259ec49bf76e561ce5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } }
I20260619 20:40:42.300217 16102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 20:40:42.300285 16076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "66a40fff02cf4633b7d99a18ef9c0288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "04c12347f09245f2aa0028b6a81c5ba9"
I20260619 20:40:42.300402 16076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 20:40:42.301252 16076 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a40fff02cf4633b7d99a18ef9c0288 in term 1.
I20260619 20:40:42.301443 16075 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9 [term 1 FOLLOWER]: Refusing update from remote peer 66a40fff02cf4633b7d99a18ef9c0288: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 20:40:42.301903 16198 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 20:40:42.302807 16238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 20:40:42.302866 16238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 20:40:42.305122 16227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66a40fff02cf4633b7d99a18ef9c0288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a40fff02cf4633b7d99a18ef9c0288" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } } peers { permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } } peers { permanent_uuid: "314acf4d1bd041259ec49bf76e561ce5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } } }
I20260619 20:40:42.305209 16227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:40:42.305261 16227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66a40fff02cf4633b7d99a18ef9c0288. Latest consensus state: current_term: 1 leader_uuid: "66a40fff02cf4633b7d99a18ef9c0288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a40fff02cf4633b7d99a18ef9c0288" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } } peers { permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } } peers { permanent_uuid: "314acf4d1bd041259ec49bf76e561ce5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } } }
I20260619 20:40:42.305316 16227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:40:42.305356 16227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "66a40fff02cf4633b7d99a18ef9c0288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a40fff02cf4633b7d99a18ef9c0288" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } } peers { permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } } peers { permanent_uuid: "314acf4d1bd041259ec49bf76e561ce5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } } }
I20260619 20:40:42.305411 16227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 20:40:42.315369 16198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "66a40fff02cf4633b7d99a18ef9c0288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a40fff02cf4633b7d99a18ef9c0288" member_type: VOTER last_known_addr { host: "127.25.62.190" port: 37963 } } peers { permanent_uuid: "04c12347f09245f2aa0028b6a81c5ba9" member_type: VOTER last_known_addr { host: "127.25.62.189" port: 40021 } } peers { permanent_uuid: "314acf4d1bd041259ec49bf76e561ce5" member_type: VOTER last_known_addr { host: "127.25.62.188" port: 44995 } } }
I20260619 20:40:42.315487 16198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288 [sys.catalog]: This master's current role is: LEADER
I20260619 20:40:42.316716 16219 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 20:40:42.317505 16219 catalog_manager.cc:1504] Loading token signing keys...
I20260619 20:40:42.321152 16219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66a40fff02cf4633b7d99a18ef9c0288: Generated new TSK 0
I20260619 20:40:42.321362 16219 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 20:40:42.353183 16030 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:42.351274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50228 (local address 127.25.62.190:37963)
0619 20:40:42.351319 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.351321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.352480 (+ 1159us) server_negotiation.cc:407] Connection header received
0619 20:40:42.352583 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.352584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.352596 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.352627 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.353113 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50228: BlockingRecv error: recv got EOF from 127.0.0.1:50228 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 20:40:42.354354 16182 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:42.352212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35460 (local address 127.25.62.188:44995)
0619 20:40:42.352252 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.352254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.353693 (+ 1439us) server_negotiation.cc:407] Connection header received
0619 20:40:42.353845 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.353847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.353856 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.353885 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.354300 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35460: BlockingRecv error: recv got EOF from 127.0.0.1:35460 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260619 20:40:42.355002 16106 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:42.351860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46678 (local address 127.25.62.189:40021)
0619 20:40:42.351903 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.351905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.353569 (+ 1664us) server_negotiation.cc:407] Connection header received
0619 20:40:42.353949 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.353951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.353962 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.353991 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.354943 (+ 952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46678: BlockingRecv error: recv got EOF from 127.0.0.1:46678 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260619 20:40:42.357499 16106 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:42.356708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46682 (local address 127.25.62.189:40021)
0619 20:40:42.356755 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.356757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.356931 (+ 174us) server_negotiation.cc:407] Connection header received
0619 20:40:42.357089 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.357090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.357109 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.357140 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.357456 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46682: BlockingRecv error: recv got EOF from 127.0.0.1:46682 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260619 20:40:42.360551 16106 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:42.359796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46696 (local address 127.25.62.189:40021)
0619 20:40:42.359829 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.359830 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.360075 (+ 245us) server_negotiation.cc:407] Connection header received
0619 20:40:42.360170 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.360171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.360181 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.360209 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.360513 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46696: BlockingRecv error: recv got EOF from 127.0.0.1:46696 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 20:40:42.364634 16106 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:42.363811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46710 (local address 127.25.62.189:40021)
0619 20:40:42.363853 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.363855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.364099 (+ 244us) server_negotiation.cc:407] Connection header received
0619 20:40:42.364198 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.364199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.364209 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.364239 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.364590 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46710: BlockingRecv error: recv got EOF from 127.0.0.1:46710 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 20:40:42.370904 16106 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:42.370050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46724 (local address 127.25.62.189:40021)
0619 20:40:42.370086 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.370088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.370348 (+ 260us) server_negotiation.cc:407] Connection header received
0619 20:40:42.370447 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.370448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.370460 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.370489 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.370858 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46724: BlockingRecv error: recv got EOF from 127.0.0.1:46724 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260619 20:40:42.381098 16106 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:42.380143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46738 (local address 127.25.62.189:40021)
0619 20:40:42.380216 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.380218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.380514 (+ 296us) server_negotiation.cc:407] Connection header received
0619 20:40:42.380614 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.380615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.380626 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.380655 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.381020 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46738: BlockingRecv error: recv got EOF from 127.0.0.1:46738 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260619 20:40:42.399322 16106 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:42.398373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46746 (local address 127.25.62.189:40021)
0619 20:40:42.398415 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.398417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.398697 (+ 280us) server_negotiation.cc:407] Connection header received
0619 20:40:42.398840 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.398841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.398854 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.398885 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.399269 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46746: BlockingRecv error: recv got EOF from 127.0.0.1:46746 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 20:40:42.433548 16106 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:42.432761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34690 (local address 127.25.62.189:40021)
0619 20:40:42.432812 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.432814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.432957 (+ 143us) server_negotiation.cc:407] Connection header received
0619 20:40:42.433052 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.433053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.433067 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.433123 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.433490 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34690: BlockingRecv error: recv got EOF from 127.0.0.1:34690 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 20:40:42.499366 16106 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:42.498690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34692 (local address 127.25.62.189:40021)
0619 20:40:42.498770 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.498772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.498849 (+ 77us) server_negotiation.cc:407] Connection header received
0619 20:40:42.498897 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.498898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.498909 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.498974 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.499315 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34692: BlockingRecv error: recv got EOF from 127.0.0.1:34692 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260619 20:40:42.629529 16106 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:42.628665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34702 (local address 127.25.62.189:40021)
0619 20:40:42.628760 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.628762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.628849 (+ 87us) server_negotiation.cc:407] Connection header received
0619 20:40:42.628951 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.628952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.628968 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.629039 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.629419 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34702: BlockingRecv error: recv got EOF from 127.0.0.1:34702 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260619 20:40:42.887689 16106 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 20:40:42.886974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34708 (local address 127.25.62.189:40021)
0619 20:40:42.887040 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:42.887042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:42.887058 (+ 16us) server_negotiation.cc:407] Connection header received
0619 20:40:42.887164 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:42.887165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:42.887178 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:42.887241 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 20:40:42.887620 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34708: BlockingRecv error: recv got EOF from 127.0.0.1:34708 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 20:40:43.252166 16195 catalog_manager.cc:1259] Loaded cluster ID: e5294225f26941bab02d17a677095ca8
I20260619 20:40:43.252269 16195 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5: loading cluster ID for follower catalog manager: success
I20260619 20:40:43.253132 16195 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5: acquiring CA information for follower catalog manager: success
I20260619 20:40:43.253453 16195 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 314acf4d1bd041259ec49bf76e561ce5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:40:43.303263 16238 catalog_manager.cc:1259] Loaded cluster ID: e5294225f26941bab02d17a677095ca8
I20260619 20:40:43.303360 16238 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9: loading cluster ID for follower catalog manager: success
I20260619 20:40:43.304167 16238 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9: acquiring CA information for follower catalog manager: success
I20260619 20:40:43.304473 16238 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 04c12347f09245f2aa0028b6a81c5ba9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 20:40:43.403801 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:43.400907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34710 (local address 127.25.62.189:40021)
0619 20:40:43.401082 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:43.401084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:43.401158 (+ 74us) server_negotiation.cc:407] Connection header received
0619 20:40:43.401258 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:43.401258 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:43.401332 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:43.401429 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:40:43.401601 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:43.402194 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:43.402988 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:43.403191 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:43.403269 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:43.403453 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:40:43.403560 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:40:43.403651 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:43.403674 (+ 23us) server_negotiation.cc:299] Negotiation successful
0619 20:40:43.403695 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
I20260619 20:40:43.408115 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:43.405637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44336 (local address 127.25.62.188:44995)
0619 20:40:43.405761 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:43.405763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:43.405772 (+ 9us) server_negotiation.cc:407] Connection header received
0619 20:40:43.405848 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:43.405849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:43.405895 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:43.405983 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:40:43.406099 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:43.406581 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:43.407308 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:43.407514 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:43.407598 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:43.407766 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:40:43.407876 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:40:43.407963 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:43.408001 (+ 38us) server_negotiation.cc:299] Negotiation successful
0619 20:40:43.408020 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":54,"threads_started":1}
I20260619 20:40:43.412791 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:43.409955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34714 (local address 127.25.62.189:40021)
0619 20:40:43.409993 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:43.409996 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:43.410200 (+ 204us) server_negotiation.cc:407] Connection header received
0619 20:40:43.410275 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:43.410276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:43.410288 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:43.410337 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:40:43.410462 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:43.410961 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:43.412023 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:43.412314 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:43.412381 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:43.412516 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:40:43.412548 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:40:43.412591 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:43.412683 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 20:40:43.412704 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 20:40:43.412837 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:43.410083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44338 (local address 127.25.62.188:44995)
0619 20:40:43.410130 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:43.410132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:43.410142 (+ 10us) server_negotiation.cc:407] Connection header received
0619 20:40:43.410166 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:43.410167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:43.410177 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:43.410226 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:40:43.410398 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:43.411139 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:43.412092 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:43.412384 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:43.412443 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:43.412570 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:40:43.412603 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:40:43.412675 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:43.412725 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 20:40:43.412745 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 20:40:43.413677 16323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 20:40:43.409848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57552 (local address 127.25.62.190:37963)
0619 20:40:43.410180 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0619 20:40:43.410182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 20:40:43.410189 (+ 7us) server_negotiation.cc:407] Connection header received
0619 20:40:43.410230 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 20:40:43.410231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 20:40:43.410281 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 20:40:43.410359 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 20:40:43.410517 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:43.411258 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:43.412418 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 20:40:43.412736 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 20:40:43.412791 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 20:40:43.413358 (+ 567us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 20:40:43.413440 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 20:40:43.413487 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 20:40:43.413542 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 20:40:43.413561 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":67,"threads_started":1}
I20260619 20:40:43.423120 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 15959
I20260619 20:40:43.428330 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 16034
I20260619 20:40:43.433255 25850 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXT8PDl/build/release/bin/kudu with pid 16109
2026-06-19T20:40:43Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2830 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5656 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (265213 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST